# Configuring PTP devices

The PTP Operator adds the `NodePtpDevice.ptp.openshift.io` custom resource definition (CRD) to OpenShift Container Platform.

When installed, the PTP Operator searches your cluster for Precision Time Protocol (PTP) capable network devices on each node. It creates and updates a `NodePtpDevice` custom resource (CR) object for each node that provides a compatible PTP-capable network device.

## Installing the PTP Operator using the CLI

As a cluster administrator, you can install the Operator by using the CLI.

Prerequisites

- A cluster installed on bare-metal hardware with nodes that have hardware that supports PTP.

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

Procedure

1. Create a namespace for the PTP Operator.

   1. Save the following YAML in the `ptp-namespace.yaml` file:

          apiVersion: v1
          kind: Namespace
          metadata:
            name: openshift-ptp
            annotations:
              workload.openshift.io/allowed: management
            labels:
              name: openshift-ptp
              openshift.io/cluster-monitoring: "true"

   2. Create the `Namespace` CR:

          $ oc create -f ptp-namespace.yaml

2. Create an Operator group for the PTP Operator.

   1. Save the following YAML in the `ptp-operatorgroup.yaml` file:

          apiVersion: operators.coreos.com/v1
          kind: OperatorGroup
          metadata:
            name: ptp-operators
            namespace: openshift-ptp
          spec:
            targetNamespaces:
            - openshift-ptp

   2. Create the `OperatorGroup` CR:

          $ oc create -f ptp-operatorgroup.yaml

3. Subscribe to the PTP Operator.

   1. Save the following YAML in the `ptp-sub.yaml` file:

          apiVersion: operators.coreos.com/v1alpha1
          kind: Subscription
          metadata:
            name: ptp-operator-subscription
            namespace: openshift-ptp
          spec:
            channel: "stable"
            name: ptp-operator
            source: redhat-operators
            sourceNamespace: openshift-marketplace

   2. Create the `Subscription` CR:

          $ oc create -f ptp-sub.yaml

4. To verify that the Operator is installed, enter the following command:

       $ oc get csv -n openshift-ptp -o custom-columns=Name:.metadata.name,Phase:.status.phase

   Example output

       Name                         Phase
       4.14.0-202301261535          Succeeded

## Installing the PTP Operator by using the web console

As a cluster administrator, you can install the PTP Operator by using the web console.

| You have to create the namespace and Operator group as mentioned in the previous section.

Procedure

1. Install the PTP Operator using the OpenShift Container Platform web console:

   1. In the OpenShift Container Platform web console, click Operators → OperatorHub.

   2. Choose PTP Operator from the list of available Operators, and then click Install.

   3. On the Install Operator page, under A specific namespace on the cluster select openshift-ptp. Then, click Install.

2. Optional: Verify that the PTP Operator installed successfully:

   1. Switch to the Operators → Installed Operators page.

   2. Ensure that PTP Operator is listed in the openshift-ptp project with a Status of InstallSucceeded.

      | During installation an Operator might display a Failed status. If the installation later succeeds with an InstallSucceeded message, you can ignore the Failed message.

      If the Operator does not appear as installed, to troubleshoot further:

      - Go to the Operators → Installed Operators page and inspect the Operator Subscriptions and Install Plans tabs for any failure or errors under Status.

      - Go to the Workloads → Pods page and check the logs for pods in the `openshift-ptp` project.

## Discovering PTP capable network devices in your cluster

- To return a complete list of PTP capable network devices in your cluster, run the following command:

      $ oc get NodePtpDevice -n openshift-ptp -o yaml

  Example output

      apiVersion: v1
      items:
      - apiVersion: ptp.openshift.io/v1
        kind: NodePtpDevice
        metadata:
          creationTimestamp: "2022-01-27T15:16:28Z"
          generation: 1
          name: dev-worker-0 (1)
          namespace: openshift-ptp
          resourceVersion: "6538103"
          uid: d42fc9ad-bcbf-4590-b6d8-b676c642781a
        spec: {}
        status:
          devices: (2)
          - name: eno1
          - name: eno2
          - name: eno3
          - name: eno4
          - name: enp5s0f0
          - name: enp5s0f1
      ...

  1 | The value for the `name` parameter is the same as the name of the parent node.
  2 | The `devices` collection includes a list of the PTP capable devices that the PTP Operator discovers for the node.

## Using hardware-specific NIC features with the PTP Operator

NIC hardware with built-in PTP capabilities sometimes require device-specific configuration. You can use hardware-specific NIC features for supported hardware with the PTP Operator by configuring a plugin in the `PtpConfig` custom resource (CR). The `linuxptp-daemon` service uses the named parameters in the `plugin` stanza to start `linuxptp` processes (`ptp4l` and `phc2sys`) based on the specific hardware configuration.

| In OpenShift Container Platform 4.14, the Intel E810 NIC is supported with a `PtpConfig` plugin.

## Configuring linuxptp services as a grandmaster clock

You can configure the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as grandmaster clock (T-GM) by creating a `PtpConfig` custom resource (CR) that configures the host NIC.

The `ts2phc` utility allows you to synchronize the system clock with the PTP grandmaster clock so that the node can stream precision clock signal to downstream PTP ordinary clocks and boundary clocks.

| Use the following example `PtpConfig` CR as the basis to configure `linuxptp` services as T-GM for an Intel Westport Channel E810-XXVDA4T network interface. To configure PTP fast events, set appropriate values for `ptp4lOpts`, `ptp4lConf`, and `ptpClockThreshold`. `ptpClockThreshold` is used only when events are enabled. See "Configuring the PTP fast event notifications publisher" for more information.

Prerequisites

- For T-GM clocks in production environments, install an Intel E810 Westport Channel NIC in the bare-metal cluster host.

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

Procedure

1. Create the `PtpConfig` CR. For example:

   1. Depending on your requirements, use one of the following T-GM configurations for your deployment. Save the YAML in the `grandmaster-clock-ptp-config.yaml` file:

      PTP grandmaster clock configuration for E810 NIC

          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: grandmaster
            namespace: openshift-ptp
            annotations: {}
          spec:
            profile:
              - name: "grandmaster"
                ptp4lOpts: "-2 --summary_interval -4"
                phc2sysOpts: -r -u 0 -m -w -N 8 -R 16 -s $iface_master -n 24
                ptpSchedulingPolicy: SCHED_FIFO
                ptpSchedulingPriority: 10
                ptpSettings:
                  logReduce: "true"
                plugins:
                  e810:
                    enableDefaultConfig: false
                    settings:
                      LocalMaxHoldoverOffSet: 1500
                      LocalHoldoverTimeout: 14400
                      MaxInSpecOffset: 100
                    pins: $e810_pins
                    #  "$iface_master":
                    #    "U.FL2": "0 2"
                    #    "U.FL1": "0 1"
                    #    "SMA2": "0 2"
                    #    "SMA1": "0 1"
                    ublxCmds:
                      - args: #ubxtool -P 29.20 -z CFG-HW-ANT_CFG_VOLTCTRL,1
                          - "-P"
                          - "29.20"
                          - "-z"
                          - "CFG-HW-ANT_CFG_VOLTCTRL,1"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -e GPS
                          - "-P"
                          - "29.20"
                          - "-e"
                          - "GPS"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d Galileo
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "Galileo"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d GLONASS
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "GLONASS"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d BeiDou
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "BeiDou"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d SBAS
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "SBAS"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -t -w 5 -v 1 -e SURVEYIN,600,50000
                          - "-P"
                          - "29.20"
                          - "-t"
                          - "-w"
                          - "5"
                          - "-v"
                          - "1"
                          - "-e"
                          - "SURVEYIN,600,50000"
                        reportOutput: true
                      - args: #ubxtool -P 29.20 -p MON-HW
                          - "-P"
                          - "29.20"
                          - "-p"
                          - "MON-HW"
                        reportOutput: true
                      - args: #ubxtool -P 29.20 -p CFG-MSG,1,38,248
                          - "-P"
                          - "29.20"
                          - "-p"
                          - "CFG-MSG,1,38,248"
                        reportOutput: true
                ts2phcOpts: " "
                ts2phcConf: |
                  [nmea]
                  ts2phc.master 1
                  [global]
                  use_syslog  0
                  verbose 1
                  logging_level 7
                  ts2phc.pulsewidth 100000000
                  #cat /dev/GNSS to find available serial port
                  #example value of gnss_serialport is /dev/ttyGNSS_1700_0
                  ts2phc.nmea_serialport $gnss_serialport
                  [$iface_master]
                  ts2phc.extts_polarity rising
                  ts2phc.extts_correction 0
                ptp4lConf: |
                  [$iface_master]
                  masterOnly 1
                  [$iface_master_1]
                  masterOnly 1
                  [$iface_master_2]
                  masterOnly 1
                  [$iface_master_3]
                  masterOnly 1
                  [global]
                  #
                  # Default Data Set
                  #
                  twoStepFlag 1
                  priority1 128
                  priority2 128
                  domainNumber 24
                  #utc_offset 37
                  clockClass 6
                  clockAccuracy 0x27
                  offsetScaledLogVariance 0xFFFF
                  free_running 0
                  freq_est_interval 1
                  dscp_event 0
                  dscp_general 0
                  dataset_comparison G.8275.x
                  G.8275.defaultDS.localPriority 128
                  #
                  # Port Data Set
                  #
                  logAnnounceInterval -3
                  logSyncInterval -4
                  logMinDelayReqInterval -4
                  logMinPdelayReqInterval 0
                  announceReceiptTimeout 3
                  syncReceiptTimeout 0
                  delayAsymmetry 0
                  fault_reset_interval -4
                  neighborPropDelayThresh 20000000
                  masterOnly 0
                  G.8275.portDS.localPriority 128
                  #
                  # Run time options
                  #
                  assume_two_step 0
                  logging_level 6
                  path_trace_enabled 0
                  follow_up_info 0
                  hybrid_e2e 0
                  inhibit_multicast_service 0
                  net_sync_monitor 0
                  tc_spanning_tree 0
                  tx_timestamp_timeout 50
                  unicast_listen 0
                  unicast_master_table 0
                  unicast_req_duration 3600
                  use_syslog 1
                  verbose 0
                  summary_interval -4
                  kernel_leap 1
                  check_fup_sync 0
                  clock_class_threshold 7
                  #
                  # Servo Options
                  #
                  pi_proportional_const 0.0
                  pi_integral_const 0.0
                  pi_proportional_scale 0.0
                  pi_proportional_exponent -0.3
                  pi_proportional_norm_max 0.7
                  pi_integral_scale 0.0
                  pi_integral_exponent 0.4
                  pi_integral_norm_max 0.3
                  step_threshold 2.0
                  first_step_threshold 0.00002
                  clock_servo pi
                  sanity_freq_limit  200000000
                  ntpshm_segment 0
                  #
                  # Transport options
                  #
                  transportSpecific 0x0
                  ptp_dst_mac 01:1B:19:00:00:00
                  p2p_dst_mac 01:80:C2:00:00:0E
                  udp_ttl 1
                  udp6_scope 0x0E
                  uds_address /var/run/ptp4l
                  #
                  # Default interface options
                  #
                  clock_type BC
                  network_transport L2
                  delay_mechanism E2E
                  time_stamping hardware
                  tsproc_mode filter
                  delay_filter moving_median
                  delay_filter_length 10
                  egressLatency 0
                  ingressLatency 0
                  boundary_clock_jbod 0
                  #
                  # Clock description
                  #
                  productDescription ;;
                  revisionData ;;
                  manufacturerIdentity 00:00:00
                  userDescription ;
                  timeSource 0x20
            recommend:
              - profile: "grandmaster"
                priority: 4
                match:
                  - nodeLabel: "node-role.kubernetes.io/$mcp"

      | For E810 Westport Channel NICs, set the value for `ts2phc.nmea_serialport` to `/dev/gnss0`.

   2. Create the CR by running the following command:

          $ oc create -f grandmaster-clock-ptp-config.yaml

Verification

1. Check that the `PtpConfig` profile is applied to the node.

   1. Get the list of pods in the `openshift-ptp` namespace by running the following command:

          $ oc get pods -n openshift-ptp -o wide

      Example output

          NAME                          READY   STATUS    RESTARTS   AGE     IP             NODE
          linuxptp-daemon-74m2g         3/3     Running   3          4d15h   10.16.230.7    compute-1.example.com
          ptp-operator-5f4f48d7c-x7zkf  1/1     Running   1          4d15h   10.128.1.145   compute-1.example.com

   2. Check that the profile is correct. Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile. Run the following command:

          $ oc logs linuxptp-daemon-74m2g -n openshift-ptp -c linuxptp-daemon-container

      Example output

          ts2phc[94980.334]: [ts2phc.0.config] nmea delay: 98690975 ns
          ts2phc[94980.334]: [ts2phc.0.config] ens3f0 extts index 0 at 1676577329.999999999 corr 0 src 1676577330.901342528 diff -1
          ts2phc[94980.334]: [ts2phc.0.config] ens3f0 master offset         -1 s2 freq      -1
          ts2phc[94980.441]: [ts2phc.0.config] nmea sentence: GNRMC,195453.00,A,4233.24427,N,07126.64420,W,0.008,,160223,,,A,V
          phc2sys[94980.450]: [ptp4l.0.config] CLOCK_REALTIME phc offset       943 s2 freq  -89604 delay    504
          phc2sys[94980.512]: [ptp4l.0.config] CLOCK_REALTIME phc offset      1000 s2 freq  -89264 delay    474

## Configuring linuxptp services as a grandmaster clock for dual E810 Westport Channel NICs

You can configure the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as grandmaster clock (T-GM) for dual E810 Westport Channel NICs by creating a `PtpConfig` custom resource (CR) that configures the host NICs.

For distributed RAN (D-RAN) use cases, you can configure PTP for dual-NICs as follows:

- NIC one is synced to the global navigation satellite system (GNSS) time source.

- NIC two is synced to the 1PPS timing output provided by NIC one. This configuration is provided by the PTP hardware plugin in the `PtpConfig` CR.

The dual-NIC PTP T-GM configuration uses a single instance of `ptp4l` and one `ts2phc` process reporting two `ts2phc` instances, one for each NIC. The host system clock is synchronized from the NIC that is connected to the GNSS time source.

| Use the following example `PtpConfig` CR as the basis to configure `linuxptp` services as T-GM for dual Intel Westport Channel E810-XXVDA4T network interfaces. To configure PTP fast events, set appropriate values for `ptp4lOpts`, `ptp4lConf`, and `ptpClockThreshold`. `ptpClockThreshold` is used only when events are enabled. See "Configuring the PTP fast event notifications publisher" for more information.

Prerequisites

- For T-GM clocks in production environments, install two Intel E810 Westport Channel NICs in the bare-metal cluster host.

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

Procedure

1. Create the `PtpConfig` CR. For example:

   1. Save the following YAML in the `grandmaster-clock-ptp-config-dual-nics.yaml` file:

      PTP grandmaster clock configuration for dual E810 NICs

          # In this example two cards $iface_nic1 and $iface_nic2 are connected via
          # SMA1 ports by a cable and $iface_nic2 receives 1PPS signals from $iface_nic1
          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: grandmaster
            namespace: openshift-ptp
            annotations: {}
          spec:
            profile:
              - name: "grandmaster"
                ptp4lOpts: "-2 --summary_interval -4"
                phc2sysOpts: -r -u 0 -m -w -N 8 -R 16 -s $iface_nic1 -n 24
                ptpSchedulingPolicy: SCHED_FIFO
                ptpSchedulingPriority: 10
                ptpSettings:
                  logReduce: "true"
                plugins:
                  e810:
                    enableDefaultConfig: false
                    settings:
                      LocalMaxHoldoverOffSet: 1500
                      LocalHoldoverTimeout: 14400
                      MaxInSpecOffset: 100
                    pins: $e810_pins
                    #  "$iface_nic1":
                    #    "U.FL2": "0 2"
                    #    "U.FL1": "0 1"
                    #    "SMA2": "0 2"
                    #    "SMA1": "2 1"
                    #  "$iface_nic2":
                    #    "U.FL2": "0 2"
                    #    "U.FL1": "0 1"
                    #    "SMA2": "0 2"
                    #    "SMA1": "1 1"
                    ublxCmds:
                      - args: #ubxtool -P 29.20 -z CFG-HW-ANT_CFG_VOLTCTRL,1
                          - "-P"
                          - "29.20"
                          - "-z"
                          - "CFG-HW-ANT_CFG_VOLTCTRL,1"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -e GPS
                          - "-P"
                          - "29.20"
                          - "-e"
                          - "GPS"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d Galileo
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "Galileo"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d GLONASS
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "GLONASS"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d BeiDou
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "BeiDou"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -d SBAS
                          - "-P"
                          - "29.20"
                          - "-d"
                          - "SBAS"
                        reportOutput: false
                      - args: #ubxtool -P 29.20 -t -w 5 -v 1 -e SURVEYIN,600,50000
                          - "-P"
                          - "29.20"
                          - "-t"
                          - "-w"
                          - "5"
                          - "-v"
                          - "1"
                          - "-e"
                          - "SURVEYIN,600,50000"
                        reportOutput: true
                      - args: #ubxtool -P 29.20 -p MON-HW
                          - "-P"
                          - "29.20"
                          - "-p"
                          - "MON-HW"
                        reportOutput: true
                      - args: #ubxtool -P 29.20 -p CFG-MSG,1,38,248
                          - "-P"
                          - "29.20"
                          - "-p"
                          - "CFG-MSG,1,38,248"
                        reportOutput: true
                ts2phcOpts: " "
                ts2phcConf: |
                  [nmea]
                  ts2phc.master 1
                  [global]
                  use_syslog  0
                  verbose 1
                  logging_level 7
                  ts2phc.pulsewidth 100000000
                  #cat /dev/GNSS to find available serial port
                  #example value of gnss_serialport is /dev/ttyGNSS_1700_0
                  ts2phc.nmea_serialport $gnss_serialport
                  [$iface_nic1]
                  ts2phc.extts_polarity rising
                  ts2phc.extts_correction 0
                  [$iface_nic2]
                  ts2phc.master 0
                  ts2phc.extts_polarity rising
                  #this is a measured value in nanoseconds to compensate for SMA cable delay
                  ts2phc.extts_correction -10
                ptp4lConf: |
                  [$iface_nic1]
                  masterOnly 1
                  [$iface_nic1_1]
                  masterOnly 1
                  [$iface_nic1_2]
                  masterOnly 1
                  [$iface_nic1_3]
                  masterOnly 1
                  [$iface_nic2]
                  masterOnly 1
                  [$iface_nic2_1]
                  masterOnly 1
                  [$iface_nic2_2]
                  masterOnly 1
                  [$iface_nic2_3]
                  masterOnly 1
                  [global]
                  #
                  # Default Data Set
                  #
                  twoStepFlag 1
                  priority1 128
                  priority2 128
                  domainNumber 24
                  #utc_offset 37
                  clockClass 6
                  clockAccuracy 0x27
                  offsetScaledLogVariance 0xFFFF
                  free_running 0
                  freq_est_interval 1
                  dscp_event 0
                  dscp_general 0
                  dataset_comparison G.8275.x
                  G.8275.defaultDS.localPriority 128
                  #
                  # Port Data Set
                  #
                  logAnnounceInterval -3
                  logSyncInterval -4
                  logMinDelayReqInterval -4
                  logMinPdelayReqInterval 0
                  announceReceiptTimeout 3
                  syncReceiptTimeout 0
                  delayAsymmetry 0
                  fault_reset_interval -4
                  neighborPropDelayThresh 20000000
                  masterOnly 0
                  G.8275.portDS.localPriority 128
                  #
                  # Run time options
                  #
                  assume_two_step 0
                  logging_level 6
                  path_trace_enabled 0
                  follow_up_info 0
                  hybrid_e2e 0
                  inhibit_multicast_service 0
                  net_sync_monitor 0
                  tc_spanning_tree 0
                  tx_timestamp_timeout 50
                  unicast_listen 0
                  unicast_master_table 0
                  unicast_req_duration 3600
                  use_syslog 1
                  verbose 0
                  summary_interval -4
                  kernel_leap 1
                  check_fup_sync 0
                  clock_class_threshold 7
                  #
                  # Servo Options
                  #
                  pi_proportional_const 0.0
                  pi_integral_const 0.0
                  pi_proportional_scale 0.0
                  pi_proportional_exponent -0.3
                  pi_proportional_norm_max 0.7
                  pi_integral_scale 0.0
                  pi_integral_exponent 0.4
                  pi_integral_norm_max 0.3
                  step_threshold 2.0
                  first_step_threshold 0.00002
                  clock_servo pi
                  sanity_freq_limit  200000000
                  ntpshm_segment 0
                  #
                  # Transport options
                  #
                  transportSpecific 0x0
                  ptp_dst_mac 01:1B:19:00:00:00
                  p2p_dst_mac 01:80:C2:00:00:0E
                  udp_ttl 1
                  udp6_scope 0x0E
                  uds_address /var/run/ptp4l
                  #
                  # Default interface options
                  #
                  clock_type BC
                  network_transport L2
                  delay_mechanism E2E
                  time_stamping hardware
                  tsproc_mode filter
                  delay_filter moving_median
                  delay_filter_length 10
                  egressLatency 0
                  ingressLatency 0
                  boundary_clock_jbod 1
                  #
                  # Clock description
                  #
                  productDescription ;;
                  revisionData ;;
                  manufacturerIdentity 00:00:00
                  userDescription ;
                  timeSource 0x20
            recommend:
              - profile: "grandmaster"
                priority: 4
                match:
                  - nodeLabel: "node-role.kubernetes.io/$mcp"

      | For E810 Westport Channel NICs, set the value for `ts2phc.nmea_serialport` to `/dev/gnss0`.

   2. Create the CR by running the following command:

          $ oc create -f grandmaster-clock-ptp-config-dual-nics.yaml

Verification

1. Check that the `PtpConfig` profile is applied to the node.

   1. Get the list of pods in the `openshift-ptp` namespace by running the following command:

          $ oc get pods -n openshift-ptp -o wide

      Example output

          NAME                          READY   STATUS    RESTARTS   AGE     IP             NODE
          linuxptp-daemon-74m2g         3/3     Running   3          4d15h   10.16.230.7    compute-1.example.com
          ptp-operator-5f4f48d7c-x7zkf  1/1     Running   1          4d15h   10.128.1.145   compute-1.example.com

   2. Check that the profile is correct. Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile. Run the following command:

          $ oc logs linuxptp-daemon-74m2g -n openshift-ptp -c linuxptp-daemon-container

      Example output

          ts2phc[509863.660]: [ts2phc.0.config] nmea delay: 347527248 ns
          ts2phc[509863.660]: [ts2phc.0.config] ens2f0 extts index 0 at 1705516553.000000000 corr 0 src 1705516553.652499081 diff 0
          ts2phc[509863.660]: [ts2phc.0.config] ens2f0 master offset          0 s2 freq      -0
          I0117 18:35:16.000146 1633226 stats.go:57] state updated for ts2phc =s2
          I0117 18:35:16.000163 1633226 event.go:417] dpll State s2, gnss State s2, tsphc state s2, gm state s2,
          ts2phc[1705516516]:[ts2phc.0.config] ens2f0 nmea_status 1 offset 0 s2
          GM[1705516516]:[ts2phc.0.config] ens2f0 T-GM-STATUS s2
          ts2phc[509863.677]: [ts2phc.0.config] ens7f0 extts index 0 at 1705516553.000000010 corr -10 src 1705516553.652499081 diff 0
          ts2phc[509863.677]: [ts2phc.0.config] ens7f0 master offset          0 s2 freq      -0
          I0117 18:35:16.016597 1633226 stats.go:57] state updated for ts2phc =s2
          phc2sys[509863.719]: [ptp4l.0.config] CLOCK_REALTIME phc offset        -6 s2 freq  +15441 delay    510
          phc2sys[509863.782]: [ptp4l.0.config] CLOCK_REALTIME phc offset        -7 s2 freq  +15438 delay    502

Additional resources

- Configuring the PTP fast event notifications publisher

### Grandmaster clock PtpConfig configuration reference

The following reference information describes the configuration options for the `PtpConfig` custom resource (CR) that configures the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as a grandmaster clock.

Table 1. PtpConfig configuration options for PTP Grandmaster clock
PtpConfig CR field | Description
--- | ---
`plugins` | Specify an array of `.exec.cmdline` options that configure the NIC for grandmaster clock operation. Grandmaster clock configuration requires certain PTP pins to be disabled. The plugin mechanism allows the PTP Operator to do automated hardware configuration. For the Intel Westport Channel NIC, when `enableDefaultConfig` is true, The PTP Operator runs a hard-coded script to do the required configuration for the NIC.
`ptp4lOpts` | Specify system configuration options for the `ptp4l` service. The options should not include the network interface name `-i <interface>` and service config file `-f /etc/ptp4l.conf` because the network interface name and the service config file are automatically appended.
`ptp4lConf` | Specify the required configuration to start `ptp4l` as a grandmaster clock. For example, the `ens2f1` interface synchronizes downstream connected devices. For grandmaster clocks, set `clockClass` to `6` and set `clockAccuracy` to `0x27`. Set `timeSource` to `0x20` for when receiving the timing signal from a Global navigation satellite system (GNSS).
`tx_timestamp_timeout` | Specify the maximum amount of time to wait for the transmit (TX) timestamp from the sender before discarding the data.
`boundary_clock_jbod` | Specify the JBOD boundary clock time delay value. This value is used to correct the time values that are passed between the network time devices.
`phc2sysOpts` | Specify system config options for the `phc2sys` service. If this field is empty the PTP Operator does not start the `phc2sys` service.
| Ensure that the network interface listed here is configured as grandmaster and is referenced as required in the `ts2phcConf` and `ptp4lConf` fields.
`ptpSchedulingPolicy` | Configure the scheduling policy for `ptp4l` and `phc2sys` processes. Default value is `SCHED_OTHER`. Use `SCHED_FIFO` on systems that support FIFO scheduling.
`ptpSchedulingPriority` | Set an integer value from 1-65 to configure FIFO priority for `ptp4l` and `phc2sys` processes when `ptpSchedulingPolicy` is set to `SCHED_FIFO`. The `ptpSchedulingPriority` field is not used when `ptpSchedulingPolicy` is set to `SCHED_OTHER`.
`ptpClockThreshold` | Optional. If `ptpClockThreshold` stanza is not present, default values are used for `ptpClockThreshold` fields. Stanza shows default `ptpClockThreshold` values. `ptpClockThreshold` values configure how long after the PTP master clock is disconnected before PTP events are triggered. `holdOverTimeout` is the time value in seconds before the PTP clock event state changes to `FREERUN` when the PTP master clock is disconnected. The `maxOffsetThreshold` and `minOffsetThreshold` settings configure offset values in nanoseconds that compare against the values for `CLOCK_REALTIME` (`phc2sys`) or master offset (`ptp4l`). When the `ptp4l` or `phc2sys` offset value is outside this range, the PTP clock state is set to `FREERUN`. When the offset value is within this range, the PTP clock state is set to `LOCKED`.
`ts2phcConf` | Sets the configuration for the `ts2phc` command. `leapfile` is the default path to the current leap seconds definition file in the PTP Operator container image. `ts2phc.nmea_serialport` is the serial port device that is connected to the NMEA GPS clock source. When configured, the GNSS receiver is accessible on `/dev/gnss<id>`. If the host has multiple GNSS receivers, you can find the correct device by enumerating either of the following devices: `/sys/class/net/<eth_port>/device/gnss/` `/sys/class/gnss/gnss<id>/device/`
`ts2phcOpts` | Set options for the `ts2phc` command.
`recommend` | Specify an array of one or more `recommend` objects that define rules on how the `profile` should be applied to nodes.
`.recommend.profile` | Specify the `.recommend.profile` object name that is defined in the `profile` section.
`.recommend.priority` | Specify the `priority` with an integer value between `0` and `99`. A larger number gets lower priority, so a priority of `99` is lower than a priority of `10`. If a node can be matched with multiple profiles according to rules defined in the `match` field, the profile with the higher priority is applied to that node.
`.recommend.match` | Specify `.recommend.match` rules with `nodeLabel` or `nodeName` values.
`.recommend.match.nodeLabel` | Set `nodeLabel` with the `key` of the `node.Labels` field from the node object by using the `oc get nodes --show-labels` command. For example, `node-role.kubernetes.io/worker`.
`.recommend.match.nodeName` | Set `nodeName` with the value of the `node.Name` field from the node object by using the `oc get nodes` command. For example, `compute-1.example.com`.

### Grandmaster clock class sync state reference

The following table describes the PTP grandmaster clock (T-GM) `gm.ClockClass` states. Clock class states categorize T-GM clocks based on their accuracy and stability with regard to the Primary Reference Time Clock (PRTC) or other timing source.

Holdover specification is the amount of time a PTP clock can maintain synchronization without receiving updates from the primary time source.

Table 2. T-GM clock class states
Clock class state | Description
--- | ---
`gm.ClockClass 6` | T-GM clock is connected to a PRTC in `LOCKED` mode. For example, the PRTC is traceable to a GNSS time source.
`gm.ClockClass 7` | T-GM clock is in `HOLDOVER` mode, and within holdover specification. The clock source might not be traceable to a category 1 frequency source.
`gm.ClockClass 140` | T-GM clock is in `HOLDOVER` mode, is out of holdover specification, but it is still traceable to the category 1 frequency source.
`gm.ClockClass 248` | T-GM clock is in `FREERUN` mode.

For more information, see "Phase/time traceability information", ITU-T G.8275.1/Y.1369.1 Recommendations.

### Intel Westport Channel E810 hardware configuration reference

Use this information to understand how to use the Intel E810-XXVDA4T hardware plugin to configure the E810 network interface as PTP grandmaster clock. Hardware pin configuration determines how the network interface interacts with other components and devices in the system. The E810-XXVDA4T NIC has four connectors for external 1PPS signals: `SMA1`, `SMA2`, `U.FL1`, and `U.FL2`.

Table 3. Intel E810 NIC hardware connectors configuration
Hardware pin | Recommended setting | Description
--- | --- | ---
`U.FL1` | `0 1` | Disables the `U.FL1` connector input. The `U.FL1` connector is output-only.
`U.FL2` | `0 2` | Disables the `U.FL2` connector output. The `U.FL2` connector is input-only.
`SMA1` | `0 1` | Disables the `SMA1` connector input. The `SMA1` connector is bidirectional.
`SMA2` | `0 2` | Disables the `SMA2` connector output. The `SMA2` connector is bidirectional.

| `SMA1` and `U.FL1` connectors share channel one. `SMA2` and `U.FL2` connectors share channel two.

Set `spec.profile.plugins.e810.ublxCmds` parameters to configure the GNSS clock in the `PtpConfig` custom resource (CR). Each of these `ublxCmds` stanzas correspond to a configuration that is applied to the host NIC by using `ubxtool` commands. For example:

    ublxCmds:
      - args: #ubxtool -P 29.20 -z CFG-HW-ANT_CFG_VOLTCTRL,1
          - "-P"
          - "29.20"
          - "-z"
          - "CFG-HW-ANT_CFG_VOLTCTRL,1"
        reportOutput: false

The following table describes the equivalent `ubxtool` commands:

Table 4. Intel E810 ublxCmds configuration
ubxtool command | Description
--- | ---
`ubxtool -P 29.20 -z CFG-HW-ANT_CFG_VOLTCTRL,1` | Enables antenna voltage control. Enables antenna status to be reported in the `UBX-MON-RF` and `UBX-INF-NOTICE` log messages.
`ubxtool -P 29.20 -e GPS` | Enables the antenna to receive GPS signals.
`ubxtool -P 29.20 -d Galileo` | Configures the antenna to receive signal from the Galileo GPS satellite.
`ubxtool -P 29.20 -d GLONASS` | Disables the antenna from receiving signal from the GLONASS GPS satellite.
`ubxtool -P 29.20 -d BeiDou` | Disables the antenna from receiving signal from the BeiDou GPS satellite.
`ubxtool -P 29.20 -d SBAS` | Disables the antenna from receiving signal from the SBAS GPS satellite.
`ubxtool -P 29.20 -t -w 5 -v 1 -e SURVEYIN,600,50000` | Configures the GNSS receiver survey-in process to improve its initial position estimate. This can take up to 24 hours to achieve an optimal result.
`ubxtool -P 29.20 -p MON-HW` | Runs a single automated scan of the hardware and reports on the NIC state and configuration settings.

The E810 plugin implements the following interfaces:

Table 5. E810 plugin interfaces
Interface | Description
--- | ---
`OnPTPConfigChangeE810` | Runs whenever you update the `PtpConfig` CR. The function parses the plugin options and applies the required configurations to the network device pins based on the configuration data.
`AfterRunPTPCommandE810` | Runs after launching the PTP processes and running the `gpspipe` PTP command. The function processes the plugin options and runs `ubxtool` commands, storing the output in the plugin-specific data.
`PopulateHwConfigE810` | Populates the `NodePtpDevice` CR based on hardware-specific data in the `PtpConfig` CR.

The E810 plugin has the following structs and variables:

Table 6. E810 plugin structs and variables
Struct | Description
--- | ---
`E810Opts` | Represents options for the E810 plugin, including boolean flags and a map of network device pins.
`E810UblxCmds` | Represents configurations for `ubxtool` commands with a boolean flag and a slice of strings for command arguments.
`E810PluginData` | Holds plugin-specific data used during plugin execution.

### Dual E810 Westport Channel NIC configuration reference

Use this information to understand how to use the Intel E810-XXVDA4T hardware plugin to configure a pair of E810 network interfaces as PTP grandmaster clock (T-GM).

Before you configure the dual-NIC cluster host, you must connect the two NICs with an SMA1 cable using the 1PPS faceplace connections.

When you configure a dual-NIC T-GM, you need to compensate for the 1PPS signal delay that occurs when you connect the NICs using the SMA1 connection ports. Various factors such as cable length, ambient temperature, and component and manufacturing tolerances can affect the signal delay. To compensate for the delay, you must calculate the specific value that you use to offset the signal delay.

Table 7. E810 dual-NIC T-GM PtpConfig CR reference
PtpConfig field | Description
--- | ---
`spec.profile.plugins.e810.pins` | Configure the E810 hardware pins using the PTP Operator E810 hardware plugin. Pin `2 1` enables the `1PPS OUT` connection for `SMA1` on NIC one. Pin `1 1` enables the `1PPS IN` connection for `SMA1` on NIC two.
`spec.profile.ts2phcConf` | Use the `ts2phcConf` field to configure parameters for NIC one and NIC two. Set `ts2phc.master 0` for NIC two. This configures the timing source for NIC two from the 1PPS input, not GNSS. Configure the `ts2phc.extts_correction` value for NIC two to compensate for the delay that is incurred for the specific SMA cable and cable length that you use. The value that you configure depends on your specific measurements and SMA1 cable length.
`spec.profile.ptp4lConf` | Set the value of `boundary_clock_jbod` to 1 to enable support for multiple NICs.

## Configuring dynamic leap seconds handling for PTP grandmaster clocks

The PTP Operator container image includes the latest `leap-seconds.list` file that is available at the time of release. You can configure the PTP Operator to automatically update the leap second file by using Global Positioning System (GPS) announcements.

Leap second information is stored in an automatically generated `ConfigMap` resource named `leap-configmap` in the `openshift-ptp` namespace. The PTP Operator mounts the `leap-configmap` resource as a volume in the `linuxptp-daemon` pod that is accessible by the `ts2phc` process.

If the GPS satellite broadcasts new leap second data, the PTP Operator updates the `leap-configmap` resource with the new data. The `ts2phc` process picks up the changes automatically.

| The following procedure is provided as reference. The 4.14 version of the PTP Operator enables automatic leap second management by default.

Prerequisites

- You have installed the OpenShift CLI (`oc`).

- You have logged in as a user with `cluster-admin` privileges.

- You have installed the PTP Operator and configured a PTP grandmaster clock (T-GM) in the cluster.

Procedure

1. Configure automatic leap second handling in the `phc2sysOpts` section of the `PtpConfig` CR. Set the following options:

       phc2sysOpts: -r -u 0 -m -w -N 8 -R 16 -S 2 -s ens2f0 -n 24 (1)

   1 | Set `-w` to force `phc2sys` to wait until `ptp4l` has synchronized the system hardware clock before starting its own synchronization process.

   | Previously, the T-GM required an offset adjustment in the `phc2sys` configuration (`-O -37`) to account for historical leap seconds. This is no longer needed.

2. Configure the Intel e810 NIC to enable periodical reporting of `NAV-TIMELS` messages by the GPS receiver in the `spec.profile.plugins.e810.ublxCmds` section of the `PtpConfig` CR. For example:

       - args: #ubxtool -P 29.20 -p CFG-MSG,1,38,248
           - "-P"
           - "29.20"
           - "-p"
           - "CFG-MSG,1,38,248"

Verification

1. Validate that the configured T-GM is receiving `NAV-TIMELS` messages from the connected GPS. Run the following command:

       $ oc -n openshift-ptp -c linuxptp-daemon-container exec -it $(oc -n openshift-ptp get pods -o name | grep daemon) -- ubxtool -t -p NAV-TIMELS -P 29.20

   Example output

       1722509534.4417
       UBX-NAV-STATUS:
         iTOW 384752000 gpsFix 5 flags 0xdd fixStat 0x0 flags2 0x8
         ttff 18261, msss 1367642864

       1722509534.4419
       UBX-NAV-TIMELS:
         iTOW 384752000 version 0 reserved2 0 0 0 srcOfCurrLs 2
         currLs 18 srcOfLsChange 2 lsChange 0 timeToLsEvent 70376866
         dateOfLsGpsWn 2441 dateOfLsGpsDn 7 reserved2 0 0 0
         valid x3

       1722509534.4421
       UBX-NAV-CLOCK:
         iTOW 384752000 clkB 784281 clkD 435 tAcc 3 fAcc 215

       1722509535.4477
       UBX-NAV-STATUS:
         iTOW 384753000 gpsFix 5 flags 0xdd fixStat 0x0 flags2 0x8
         ttff 18261, msss 1367643864

       1722509535.4479
       UBX-NAV-CLOCK:
         iTOW 384753000 clkB 784716 clkD 435 tAcc 3 fAcc 218

2. Validate that the `leap-configmap` resource has been successfully generated by the PTP Operator and is up to date with the latest version of the leap-seconds.list. Run the following command:

       $ oc -n openshift-ptp get configmap leap-configmap -o jsonpath='{.data.<node_name>}' (1)

   1 | Replace `<node_name>` with the node where you have installed and configured the PTP T-GM clock with automatic leap second management. Escape special characters in the node name. For example, `node-1\.example\.com`.

   Example output

       # Do not edit
       # This file is generated automatically by linuxptp-daemon
       #$  3913697179
       #@  4291747200
       2272060800     10    # 1 Jan 1972
       2287785600     11    # 1 Jul 1972
       2303683200     12    # 1 Jan 1973
       2335219200     13    # 1 Jan 1974
       2366755200     14    # 1 Jan 1975
       2398291200     15    # 1 Jan 1976
       2429913600     16    # 1 Jan 1977
       2461449600     17    # 1 Jan 1978
       2492985600     18    # 1 Jan 1979
       2524521600     19    # 1 Jan 1980
       2571782400     20    # 1 Jul 1981
       2603318400     21    # 1 Jul 1982
       2634854400     22    # 1 Jul 1983
       2698012800     23    # 1 Jul 1985
       2776982400     24    # 1 Jan 1988
       2840140800     25    # 1 Jan 1990
       2871676800     26    # 1 Jan 1991
       2918937600     27    # 1 Jul 1992
       2950473600     28    # 1 Jul 1993
       2982009600     29    # 1 Jul 1994
       3029443200     30    # 1 Jan 1996
       3076704000     31    # 1 Jul 1997
       3124137600     32    # 1 Jan 1999
       3345062400     33    # 1 Jan 2006
       3439756800     34    # 1 Jan 2009
       3550089600     35    # 1 Jul 2012
       3644697600     36    # 1 Jul 2015
       3692217600     37    # 1 Jan 2017

       #h  e65754d4 8f39962b aa854a61 661ef546 d2af0bfa

## Configuring linuxptp services as a boundary clock

You can configure the `linuxptp` services (`ptp4l`, `phc2sys`) as boundary clock by creating a `PtpConfig` custom resource (CR) object.

| Use the following example `PtpConfig` CR as the basis to configure `linuxptp` services as the boundary clock for your particular hardware and environment. This example CR does not configure PTP fast events. To configure PTP fast events, set appropriate values for `ptp4lOpts`, `ptp4lConf`, and `ptpClockThreshold`. `ptpClockThreshold` is used only when events are enabled. See "Configuring the PTP fast event notifications publisher" for more information.

Prerequisites

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

Procedure

1. Create the following `PtpConfig` CR, and then save the YAML in the `boundary-clock-ptp-config.yaml` file.

   Example PTP boundary clock configuration

       apiVersion: ptp.openshift.io/v1
       kind: PtpConfig
       metadata:
         name: boundary-clock
         namespace: openshift-ptp
         annotations: {}
       spec:
         profile:
           - name: boundary-clock
             ptp4lOpts: "-2"
             phc2sysOpts: "-a -r -n 24"
             ptpSchedulingPolicy: SCHED_FIFO
             ptpSchedulingPriority: 10
             ptpSettings:
               logReduce: "true"
             ptp4lConf: |
               # The interface name is hardware-specific
               [$iface_slave]
               masterOnly 0
               [$iface_master_1]
               masterOnly 1
               [$iface_master_2]
               masterOnly 1
               [$iface_master_3]
               masterOnly 1
               [global]
               #
               # Default Data Set
               #
               twoStepFlag 1
               slaveOnly 0
               priority1 128
               priority2 128
               domainNumber 24
               #utc_offset 37
               clockClass 248
               clockAccuracy 0xFE
               offsetScaledLogVariance 0xFFFF
               free_running 0
               freq_est_interval 1
               dscp_event 0
               dscp_general 0
               dataset_comparison G.8275.x
               G.8275.defaultDS.localPriority 128
               #
               # Port Data Set
               #
               logAnnounceInterval -3
               logSyncInterval -4
               logMinDelayReqInterval -4
               logMinPdelayReqInterval -4
               announceReceiptTimeout 3
               syncReceiptTimeout 0
               delayAsymmetry 0
               fault_reset_interval -4
               neighborPropDelayThresh 20000000
               masterOnly 0
               G.8275.portDS.localPriority 128
               #
               # Run time options
               #
               assume_two_step 0
               logging_level 6
               path_trace_enabled 0
               follow_up_info 0
               hybrid_e2e 0
               inhibit_multicast_service 0
               net_sync_monitor 0
               tc_spanning_tree 0
               tx_timestamp_timeout 50
               unicast_listen 0
               unicast_master_table 0
               unicast_req_duration 3600
               use_syslog 1
               verbose 0
               summary_interval 0
               kernel_leap 1
               check_fup_sync 0
               clock_class_threshold 135
               #
               # Servo Options
               #
               pi_proportional_const 0.0
               pi_integral_const 0.0
               pi_proportional_scale 0.0
               pi_proportional_exponent -0.3
               pi_proportional_norm_max 0.7
               pi_integral_scale 0.0
               pi_integral_exponent 0.4
               pi_integral_norm_max 0.3
               step_threshold 2.0
               first_step_threshold 0.00002
               max_frequency 900000000
               clock_servo pi
               sanity_freq_limit 200000000
               ntpshm_segment 0
               #
               # Transport options
               #
               transportSpecific 0x0
               ptp_dst_mac 01:1B:19:00:00:00
               p2p_dst_mac 01:80:C2:00:00:0E
               udp_ttl 1
               udp6_scope 0x0E
               uds_address /var/run/ptp4l
               #
               # Default interface options
               #
               clock_type BC
               network_transport L2
               delay_mechanism E2E
               time_stamping hardware
               tsproc_mode filter
               delay_filter moving_median
               delay_filter_length 10
               egressLatency 0
               ingressLatency 0
               boundary_clock_jbod 0
               #
               # Clock description
               #
               productDescription ;;
               revisionData ;;
               manufacturerIdentity 00:00:00
               userDescription ;
               timeSource 0xA0
         recommend:
           - profile: boundary-clock
             priority: 4
             match:
               - nodeLabel: "node-role.kubernetes.io/$mcp"

   Table 8. PTP boundary clock CR configuration options
   CR field | Description
   --- | ---
   `name` | The name of the `PtpConfig` CR.
   `profile` | Specify an array of one or more `profile` objects.
   `name` | Specify the name of a profile object which uniquely identifies a profile object.
   `ptp4lOpts` | Specify system config options for the `ptp4l` service. The options should not include the network interface name `-i <interface>` and service config file `-f /etc/ptp4l.conf` because the network interface name and the service config file are automatically appended.
   `ptp4lConf` | Specify the required configuration to start `ptp4l` as boundary clock. For example, `ens1f0` synchronizes from a grandmaster clock and `ens1f3` synchronizes connected devices.
   `<interface_1>` | The interface that receives the synchronization clock.
   `<interface_2>` | The interface that sends the synchronization clock.
   `tx_timestamp_timeout` | For Intel Columbiaville 800 Series NICs, set `tx_timestamp_timeout` to `50`.
   `boundary_clock_jbod` | For Intel Columbiaville 800 Series NICs, ensure `boundary_clock_jbod` is set to `0`. For Intel Fortville X710 Series NICs, ensure `boundary_clock_jbod` is set to `1`.
   `phc2sysOpts` | Specify system config options for the `phc2sys` service. If this field is empty, the PTP Operator does not start the `phc2sys` service.
   `ptpSchedulingPolicy` | Scheduling policy for ptp4l and phc2sys processes. Default value is `SCHED_OTHER`. Use `SCHED_FIFO` on systems that support FIFO scheduling.
   `ptpSchedulingPriority` | Integer value from 1-65 used to set FIFO priority for `ptp4l` and `phc2sys` processes when `ptpSchedulingPolicy` is set to `SCHED_FIFO`. The `ptpSchedulingPriority` field is not used when `ptpSchedulingPolicy` is set to `SCHED_OTHER`.
   `ptpClockThreshold` | Optional. If `ptpClockThreshold` is not present, default values are used for the `ptpClockThreshold` fields. `ptpClockThreshold` configures how long after the PTP master clock is disconnected before PTP events are triggered. `holdOverTimeout` is the time value in seconds before the PTP clock event state changes to `FREERUN` when the PTP master clock is disconnected. The `maxOffsetThreshold` and `minOffsetThreshold` settings configure offset values in nanoseconds that compare against the values for `CLOCK_REALTIME` (`phc2sys`) or master offset (`ptp4l`). When the `ptp4l` or `phc2sys` offset value is outside this range, the PTP clock state is set to `FREERUN`. When the offset value is within this range, the PTP clock state is set to `LOCKED`.
   `recommend` | Specify an array of one or more `recommend` objects that define rules on how the `profile` should be applied to nodes.
   `.recommend.profile` | Specify the `.recommend.profile` object name defined in the `profile` section.
   `.recommend.priority` | Specify the `priority` with an integer value between `0` and `99`. A larger number gets lower priority, so a priority of `99` is lower than a priority of `10`. If a node can be matched with multiple profiles according to rules defined in the `match` field, the profile with the higher priority is applied to that node.
   `.recommend.match` | Specify `.recommend.match` rules with `nodeLabel` or `nodeName` values.
   `.recommend.match.nodeLabel` | Set `nodeLabel` with the `key` of the `node.Labels` field from the node object by using the `oc get nodes --show-labels` command. For example, `node-role.kubernetes.io/worker`.
   `.recommend.match.nodeName` | Set `nodeName` with the value of the `node.Name` field from the node object by using the `oc get nodes` command. For example, `compute-1.example.com`.

2. Create the CR by running the following command:

       $ oc create -f boundary-clock-ptp-config.yaml

Verification

1. Check that the `PtpConfig` profile is applied to the node.

   1. Get the list of pods in the `openshift-ptp` namespace by running the following command:

          $ oc get pods -n openshift-ptp -o wide

      Example output

          NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE
          linuxptp-daemon-4xkbb           1/1     Running   0          43m   10.1.196.24      compute-0.example.com
          linuxptp-daemon-tdspf           1/1     Running   0          43m   10.1.196.25      compute-1.example.com
          ptp-operator-657bbb64c8-2f8sj   1/1     Running   0          43m   10.129.0.61      control-plane-1.example.com

   2. Check that the profile is correct. Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile. Run the following command:

          $ oc logs linuxptp-daemon-4xkbb -n openshift-ptp -c linuxptp-daemon-container

      Example output

          I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
          I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
          I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
          I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: profile1
          I1115 09:41:17.117616 4143292 daemon.go:102] Interface:
          I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -2
          I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r -n 24
          I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------

Additional resources

- Configuring FIFO priority scheduling for PTP hardware

- Configuring the PTP fast event notifications publisher

### Configuring linuxptp services as boundary clocks for dual-NIC hardware

You can configure the `linuxptp` services (`ptp4l`, `phc2sys`) as boundary clocks for dual-NIC hardware by creating a `PtpConfig` custom resource (CR) object for each NIC.

Dual NIC hardware allows you to connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.

Prerequisites

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

Procedure

1. Create two separate `PtpConfig` CRs, one for each NIC, using the reference CR in "Configuring linuxptp services as a boundary clock" as the basis for each CR. For example:

   1. Create `boundary-clock-ptp-config-nic1.yaml`, specifying values for `phc2sysOpts`:

          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: boundary-clock-ptp-config-nic1
            namespace: openshift-ptp
          spec:
            profile:
            - name: "profile1"
              ptp4lOpts: "-2 --summary_interval -4"
              ptp4lConf: | (1)
                [ens5f1]
                masterOnly 1
                [ens5f0]
                masterOnly 0
              ...
              phc2sysOpts: "-a -r -m -n 24 -N 8 -R 16" (2)

      1 | Specify the required interfaces to start `ptp4l` as a boundary clock. For example, `ens5f0` synchronizes from a grandmaster clock and `ens5f1` synchronizes connected devices.
      2 | Required `phc2sysOpts` values. `-m` prints messages to `stdout`. The `linuxptp-daemon` `DaemonSet` parses the logs and generates Prometheus metrics.

   2. Create `boundary-clock-ptp-config-nic2.yaml`, removing the `phc2sysOpts` field altogether to disable the `phc2sys` service for the second NIC:

          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: boundary-clock-ptp-config-nic2
            namespace: openshift-ptp
          spec:
            profile:
            - name: "profile2"
              ptp4lOpts: "-2 --summary_interval -4"
              ptp4lConf: | (1)
                [ens7f1]
                masterOnly 1
                [ens7f0]
                masterOnly 0
          ...

      1 | Specify the required interfaces to start `ptp4l` as a boundary clock on the second NIC.

      | You must completely remove the `phc2sysOpts` field from the second `PtpConfig` CR to disable the `phc2sys` service on the second NIC.

2. Create the dual-NIC `PtpConfig` CRs by running the following commands:

   1. Create the CR that configures PTP for the first NIC:

          $ oc create -f boundary-clock-ptp-config-nic1.yaml

   2. Create the CR that configures PTP for the second NIC:

          $ oc create -f boundary-clock-ptp-config-nic2.yaml

Verification

- Check that the PTP Operator has applied the `PtpConfig` CRs for both NICs. Examine the logs for the `linuxptp` daemon corresponding to the node that has the dual-NIC hardware installed. For example, run the following command:

      $ oc logs linuxptp-daemon-cvgr6 -n openshift-ptp -c linuxptp-daemon-container

  Example output

      ptp4l[80828.335]: [ptp4l.1.config] master offset          5 s2 freq   -5727 path delay       519
      ptp4l[80828.343]: [ptp4l.0.config] master offset         -5 s2 freq  -10607 path delay       533
      phc2sys[80828.390]: [ptp4l.0.config] CLOCK_REALTIME phc offset         1 s2 freq  -87239 delay    539

### Configuring linuxptp as a highly available system clock for dual-NIC Intel E810 PTP boundary clocks

You can configure the `linuxptp` services `ptp4l` and `phc2sys` as a highly available (HA) system clock for dual PTP boundary clocks (T-BC).

The highly available system clock uses multiple time sources from dual-NIC Intel E810 Salem channel hardware configured as two boundary clocks. Two boundary clocks instances participate in the HA setup, each with its own configuration profile. You connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.

Create two `PtpConfig` custom resource (CR) objects that configure the NICs as T-BC and a third `PtpConfig` CR that configures high availability between the two NICs.

| You set `phc2SysOpts` options once in the `PtpConfig` CR that configures HA. Set the `phc2sysOpts` field to an empty string in the `PtpConfig` CRs that configure the two NICs. This prevents individual `phc2sys` processes from being set up for the two profiles.

The third `PtpConfig` CR configures a highly available system clock service. The CR sets the `ptp4lOpts` field to an empty string to prevent the `ptp4l` process from running. The CR adds profiles for the `ptp4l` configurations under the `spec.profile.ptpSettings.haProfiles` key and passes the kernel socket path of those profiles to the `phc2sys` service. When a `ptp4l` failure occurs, the `phc2sys` service switches to the backup `ptp4l` configuration. When the primary profile becomes active again, the `phc2sys` service reverts to the original state.

| Ensure that you set `spec.recommend.priority` to the same value for all three `PtpConfig` CRs that you use to configure HA.

Prerequisites

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

- Configure a cluster node with Intel E810 Salem channel dual-NIC.

Procedure

1. Create two separate `PtpConfig` CRs, one for each NIC, using the CRs in "Configuring linuxptp services as boundary clocks for dual-NIC hardware" as a reference for each CR.

   1. Create the `ha-ptp-config-nic1.yaml` file, specifying an empty string for the `phc2sysOpts` field. For example:

          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: ha-ptp-config-nic1
            namespace: openshift-ptp
          spec:
            profile:
            - name: "ha-ptp-config-profile1"
              ptp4lOpts: "-2 --summary_interval -4"
              ptp4lConf: | (1)
                [ens5f1]
                masterOnly 1
                [ens5f0]
                masterOnly 0
              #...
              phc2sysOpts: "" (2)

      1 | Specify the required interfaces to start `ptp4l` as a boundary clock. For example, `ens5f0` synchronizes from a grandmaster clock and `ens5f1` synchronizes connected devices.
      2 | Set `phc2sysOpts` with an empty string. These values are populated from the `spec.profile.ptpSettings.haProfiles` field of the `PtpConfig` CR that configures high availability.

   2. Apply the `PtpConfig` CR for NIC 1 by running the following command:

          $ oc create -f ha-ptp-config-nic1.yaml

   3. Create the `ha-ptp-config-nic2.yaml` file, specifying an empty string for the `phc2sysOpts` field. For example:

          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: ha-ptp-config-nic2
            namespace: openshift-ptp
          spec:
            profile:
            - name: "ha-ptp-config-profile2"
              ptp4lOpts: "-2 --summary_interval -4"
              ptp4lConf: |
                [ens7f1]
                masterOnly 1
                [ens7f0]
                masterOnly 0
              #...
              phc2sysOpts: ""

   4. Apply the `PtpConfig` CR for NIC 2 by running the following command:

          $ oc create -f ha-ptp-config-nic2.yaml

2. Create the `PtpConfig` CR that configures the HA system clock. For example:

   1. Create the `ptp-config-for-ha.yaml` file. Set `haProfiles` to match the `metadata.name` fields that are set in the `PtpConfig` CRs that configure the two NICs. For example: `haProfiles: ha-ptp-config-nic1,ha-ptp-config-nic2`

          apiVersion: ptp.openshift.io/v1
          kind: PtpConfig
          metadata:
            name: boundary-ha
            namespace: openshift-ptp
            annotations: {}
          spec:
            profile:
              - name: "boundary-ha"
                ptp4lOpts: "" (1)
                phc2sysOpts: "-a -r -n 24"
                ptpSchedulingPolicy: SCHED_FIFO
                ptpSchedulingPriority: 10
                ptpSettings:
                  logReduce: "true"
                  haProfiles: "$profile1,$profile2"
            recommend:
              - profile: "boundary-ha"
                priority: 4
                match:
                  - nodeLabel: "node-role.kubernetes.io/$mcp"

      1 | Set the `ptp4lOpts` field to an empty string. If it is not empty, the `p4ptl` process starts with a critical error.

   | Do not apply the high availability `PtpConfig` CR before the `PtpConfig` CRs that configure the individual NICs.

   1. Apply the HA `PtpConfig` CR by running the following command:

          $ oc create -f ptp-config-for-ha.yaml

Verification

- Verify that the PTP Operator has applied the `PtpConfig` CRs correctly. Perform the following steps:

  1. Get the list of pods in the `openshift-ptp` namespace by running the following command:

         $ oc get pods -n openshift-ptp -o wide

     Example output

         NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE
         linuxptp-daemon-4xkrb           1/1     Running   0          43m   10.1.196.24      compute-0.example.com
         ptp-operator-657bbq64c8-2f8sj   1/1     Running   0          43m   10.129.0.61      control-plane-1.example.com

     | There should be only one `linuxptp-daemon` pod.

  2. Check that the profile is correct by running the following command. Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile.

         $ oc logs linuxptp-daemon-4xkrb -n openshift-ptp -c linuxptp-daemon-container

     Example output

         I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
         I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
         I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
         I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: ha-ptp-config-profile1
         I1115 09:41:17.117616 4143292 daemon.go:102] Interface:
         I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -2
         I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r -n 24
         I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------

## Configuring linuxptp services as an ordinary clock

You can configure `linuxptp` services (`ptp4l`, `phc2sys`) as ordinary clock by creating a `PtpConfig` custom resource (CR) object.

| Use the following example `PtpConfig` CR as the basis to configure `linuxptp` services as an ordinary clock for your particular hardware and environment. This example CR does not configure PTP fast events. To configure PTP fast events, set appropriate values for `ptp4lOpts`, `ptp4lConf`, and `ptpClockThreshold`. `ptpClockThreshold` is required only when events are enabled. See "Configuring the PTP fast event notifications publisher" for more information.

Prerequisites

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

Procedure

1. Create the following `PtpConfig` CR, and then save the YAML in the `ordinary-clock-ptp-config.yaml` file.

   Example PTP ordinary clock configuration

       apiVersion: ptp.openshift.io/v1
       kind: PtpConfig
       metadata:
         name: ordinary-clock
         namespace: openshift-ptp
         annotations: {}
       spec:
         profile:
           - name: ordinary-clock
             # The interface name is hardware-specific
             interface: $interface
             ptp4lOpts: "-2 -s"
             phc2sysOpts: "-a -r -n 24"
             ptpSchedulingPolicy: SCHED_FIFO
             ptpSchedulingPriority: 10
             ptpSettings:
               logReduce: "true"
             ptp4lConf: |
               [global]
               #
               # Default Data Set
               #
               twoStepFlag 1
               slaveOnly 1
               priority1 128
               priority2 128
               domainNumber 24
               #utc_offset 37
               clockClass 255
               clockAccuracy 0xFE
               offsetScaledLogVariance 0xFFFF
               free_running 0
               freq_est_interval 1
               dscp_event 0
               dscp_general 0
               dataset_comparison G.8275.x
               G.8275.defaultDS.localPriority 128
               #
               # Port Data Set
               #
               logAnnounceInterval -3
               logSyncInterval -4
               logMinDelayReqInterval -4
               logMinPdelayReqInterval -4
               announceReceiptTimeout 3
               syncReceiptTimeout 0
               delayAsymmetry 0
               fault_reset_interval -4
               neighborPropDelayThresh 20000000
               masterOnly 0
               G.8275.portDS.localPriority 128
               #
               # Run time options
               #
               assume_two_step 0
               logging_level 6
               path_trace_enabled 0
               follow_up_info 0
               hybrid_e2e 0
               inhibit_multicast_service 0
               net_sync_monitor 0
               tc_spanning_tree 0
               tx_timestamp_timeout 50
               unicast_listen 0
               unicast_master_table 0
               unicast_req_duration 3600
               use_syslog 1
               verbose 0
               summary_interval 0
               kernel_leap 1
               check_fup_sync 0
               clock_class_threshold 7
               #
               # Servo Options
               #
               pi_proportional_const 0.0
               pi_integral_const 0.0
               pi_proportional_scale 0.0
               pi_proportional_exponent -0.3
               pi_proportional_norm_max 0.7
               pi_integral_scale 0.0
               pi_integral_exponent 0.4
               pi_integral_norm_max 0.3
               step_threshold 2.0
               first_step_threshold 0.00002
               max_frequency 900000000
               clock_servo pi
               sanity_freq_limit 200000000
               ntpshm_segment 0
               #
               # Transport options
               #
               transportSpecific 0x0
               ptp_dst_mac 01:1B:19:00:00:00
               p2p_dst_mac 01:80:C2:00:00:0E
               udp_ttl 1
               udp6_scope 0x0E
               uds_address /var/run/ptp4l
               #
               # Default interface options
               #
               clock_type OC
               network_transport L2
               delay_mechanism E2E
               time_stamping hardware
               tsproc_mode filter
               delay_filter moving_median
               delay_filter_length 10
               egressLatency 0
               ingressLatency 0
               boundary_clock_jbod 0
               #
               # Clock description
               #
               productDescription ;;
               revisionData ;;
               manufacturerIdentity 00:00:00
               userDescription ;
               timeSource 0xA0
         recommend:
           - profile: ordinary-clock
             priority: 4
             match:
               - nodeLabel: "node-role.kubernetes.io/$mcp"

   Table 9. PTP ordinary clock CR configuration options
   CR field | Description
   --- | ---
   `name` | The name of the `PtpConfig` CR.
   `profile` | Specify an array of one or more `profile` objects. Each profile must be uniquely named.
   `interface` | Specify the network interface to be used by the `ptp4l` service, for example `ens787f1`.
   `ptp4lOpts` | Specify system config options for the `ptp4l` service, for example `-2` to select the IEEE 802.3 network transport. The options should not include the network interface name `-i <interface>` and service config file `-f /etc/ptp4l.conf` because the network interface name and the service config file are automatically appended. Append `--summary_interval -4` to use PTP fast events with this interface.
   `phc2sysOpts` | Specify system config options for the `phc2sys` service. If this field is empty, the PTP Operator does not start the `phc2sys` service. For Intel Columbiaville 800 Series NICs, set `phc2sysOpts` options to `-a -r -m -n 24 -N 8 -R 16`. `-m` prints messages to `stdout`. The `linuxptp-daemon` `DaemonSet` parses the logs and generates Prometheus metrics.
   `ptp4lConf` | Specify a string that contains the configuration to replace the default `/etc/ptp4l.conf` file. To use the default configuration, leave the field empty.
   `tx_timestamp_timeout` | For Intel Columbiaville 800 Series NICs, set `tx_timestamp_timeout` to `50`.
   `boundary_clock_jbod` | For Intel Columbiaville 800 Series NICs, set `boundary_clock_jbod` to `0`.
   `ptpSchedulingPolicy` | Scheduling policy for `ptp4l` and `phc2sys` processes. Default value is `SCHED_OTHER`. Use `SCHED_FIFO` on systems that support FIFO scheduling.
   `ptpSchedulingPriority` | Integer value from 1-65 used to set FIFO priority for `ptp4l` and `phc2sys` processes when `ptpSchedulingPolicy` is set to `SCHED_FIFO`. The `ptpSchedulingPriority` field is not used when `ptpSchedulingPolicy` is set to `SCHED_OTHER`.
   `ptpClockThreshold` | Optional. If `ptpClockThreshold` is not present, default values are used for the `ptpClockThreshold` fields. `ptpClockThreshold` configures how long after the PTP master clock is disconnected before PTP events are triggered. `holdOverTimeout` is the time value in seconds before the PTP clock event state changes to `FREERUN` when the PTP master clock is disconnected. The `maxOffsetThreshold` and `minOffsetThreshold` settings configure offset values in nanoseconds that compare against the values for `CLOCK_REALTIME` (`phc2sys`) or master offset (`ptp4l`). When the `ptp4l` or `phc2sys` offset value is outside this range, the PTP clock state is set to `FREERUN`. When the offset value is within this range, the PTP clock state is set to `LOCKED`.
   `recommend` | Specify an array of one or more `recommend` objects that define rules on how the `profile` should be applied to nodes.
   `.recommend.profile` | Specify the `.recommend.profile` object name defined in the `profile` section.
   `.recommend.priority` | Set `.recommend.priority` to `0` for ordinary clock.
   `.recommend.match` | Specify `.recommend.match` rules with `nodeLabel` or `nodeName` values.
   `.recommend.match.nodeLabel` | Set `nodeLabel` with the `key` of the `node.Labels` field from the node object by using the `oc get nodes --show-labels` command. For example, `node-role.kubernetes.io/worker`.
   `.recommend.match.nodeName` | Set `nodeName` with the value of the `node.Name` field from the node object by using the `oc get nodes` command. For example, `compute-1.example.com`.

2. Create the `PtpConfig` CR by running the following command:

       $ oc create -f ordinary-clock-ptp-config.yaml

Verification

1. Check that the `PtpConfig` profile is applied to the node.

   1. Get the list of pods in the `openshift-ptp` namespace by running the following command:

          $ oc get pods -n openshift-ptp -o wide

      Example output

          NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE
          linuxptp-daemon-4xkbb           1/1     Running   0          43m   10.1.196.24      compute-0.example.com
          linuxptp-daemon-tdspf           1/1     Running   0          43m   10.1.196.25      compute-1.example.com
          ptp-operator-657bbb64c8-2f8sj   1/1     Running   0          43m   10.129.0.61      control-plane-1.example.com

   2. Check that the profile is correct. Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile. Run the following command:

          $ oc logs linuxptp-daemon-4xkbb -n openshift-ptp -c linuxptp-daemon-container

      Example output

          I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
          I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
          I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
          I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: profile1
          I1115 09:41:17.117616 4143292 daemon.go:102] Interface: ens787f1
          I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -2 -s
          I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r -n 24
          I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------

Additional resources

- Configuring FIFO priority scheduling for PTP hardware

- Configuring the PTP fast event notifications publisher

### Intel Columbiaville E800 series NIC as PTP ordinary clock reference

The following table describes the changes that you must make to the reference PTP configuration to use Intel Columbiaville E800 series NICs as ordinary clocks. Make the changes in a `PtpConfig` custom resource (CR) that you apply to the cluster.

Table 10. Recommended PTP settings for Intel Columbiaville NIC
PTP configuration | Recommended setting
--- | ---
`phc2sysOpts` | `-a -r -m -n 24 -N 8 -R 16`
`tx_timestamp_timeout` | `50`
`boundary_clock_jbod` | `0`

| For `phc2sysOpts`, `-m` prints messages to `stdout`. The `linuxptp-daemon` `DaemonSet` parses the logs and generates Prometheus metrics.

Additional resources

- For a complete example CR that configures `linuxptp` services as an ordinary clock with PTP fast events, see Configuring linuxptp services as ordinary clock.

## Configuring FIFO priority scheduling for PTP hardware

In telco or other deployment types that require low latency performance, PTP daemon threads run in a constrained CPU footprint alongside the rest of the infrastructure components. By default, PTP threads run with the `SCHED_OTHER` policy. Under high load, these threads might not get the scheduling latency they require for error-free operation.

To mitigate against potential scheduling latency errors, you can configure the PTP Operator `linuxptp` services to allow threads to run with a `SCHED_FIFO` policy. If `SCHED_FIFO` is set for a `PtpConfig` CR, then `ptp4l` and `phc2sys` will run in the parent container under `chrt` with a priority set by the `ptpSchedulingPriority` field of the `PtpConfig` CR.

| Setting `ptpSchedulingPolicy` is optional, and is only required if you are experiencing latency errors.

Procedure

1. Edit the `PtpConfig` CR profile:

       $ oc edit PtpConfig -n openshift-ptp

2. Change the `ptpSchedulingPolicy` and `ptpSchedulingPriority` fields:

       apiVersion: ptp.openshift.io/v1
       kind: PtpConfig
       metadata:
         name: <ptp_config_name>
         namespace: openshift-ptp
       ...
       spec:
         profile:
         - name: "profile1"
       ...
           ptpSchedulingPolicy: SCHED_FIFO (1)
           ptpSchedulingPriority: 10 (2)

   1 | Scheduling policy for `ptp4l` and `phc2sys` processes. Use `SCHED_FIFO` on systems that support FIFO scheduling.
   2 | Required. Sets the integer value 1-65 used to configure FIFO priority for `ptp4l` and `phc2sys` processes.

3. Save and exit to apply the changes to the `PtpConfig` CR.

Verification

1. Get the name of the `linuxptp-daemon` pod and corresponding node where the `PtpConfig` CR has been applied:

       $ oc get pods -n openshift-ptp -o wide

   Example output

       NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE
       linuxptp-daemon-gmv2n           3/3     Running   0          1d17h   10.1.196.24   compute-0.example.com
       linuxptp-daemon-lgm55           3/3     Running   0          1d17h   10.1.196.25   compute-1.example.com
       ptp-operator-3r4dcvf7f4-zndk7   1/1     Running   0          1d7h    10.129.0.61   control-plane-1.example.com

2. Check that the `ptp4l` process is running with the updated `chrt` FIFO priority:

       $ oc -n openshift-ptp logs linuxptp-daemon-lgm55 -c linuxptp-daemon-container|grep chrt

   Example output

       I1216 19:24:57.091872 1600715 daemon.go:285] /bin/chrt -f 65 /usr/sbin/ptp4l -f /var/run/ptp4l.0.config -2  --summary_interval -4 -m

## Configuring log filtering for linuxptp services

The `linuxptp` daemon generates logs that you can use for debugging purposes. In telco or other deployment types that feature a limited storage capacity, these logs can add to the storage demand.

To reduce the number log messages, you can configure the `PtpConfig` custom resource (CR) to exclude log messages that report the `master offset` value. The `master offset` log message reports the difference between the current node’s clock and the master clock in nanoseconds.

Prerequisites

- Install the OpenShift CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator.

Procedure

1. Edit the `PtpConfig` CR:

       $ oc edit PtpConfig -n openshift-ptp

2. In `spec.profile`, add the `ptpSettings.logReduce` specification and set the value to `true`:

       apiVersion: ptp.openshift.io/v1
       kind: PtpConfig
       metadata:
         name: <ptp_config_name>
         namespace: openshift-ptp
       ...
       spec:
         profile:
         - name: "profile1"
       ...
           ptpSettings:
             logReduce: "true"

   | For debugging purposes, you can revert this specification to `False` to include the master offset messages.

3. Save and exit to apply the changes to the `PtpConfig` CR.

Verification

1. Get the name of the `linuxptp-daemon` pod and corresponding node where the `PtpConfig` CR has been applied:

       $ oc get pods -n openshift-ptp -o wide

   Example output

       NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE
       linuxptp-daemon-gmv2n           3/3     Running   0          1d17h   10.1.196.24   compute-0.example.com
       linuxptp-daemon-lgm55           3/3     Running   0          1d17h   10.1.196.25   compute-1.example.com
       ptp-operator-3r4dcvf7f4-zndk7   1/1     Running   0          1d7h    10.129.0.61   control-plane-1.example.com

2. Verify that master offset messages are excluded from the logs by running the following command:

       $ oc -n openshift-ptp logs <linux_daemon_container> -c linuxptp-daemon-container | grep "master offset" (1)

   1 | <linux_daemon_container> is the name of the `linuxptp-daemon` pod, for example `linuxptp-daemon-gmv2n`.

   When you configure the `logReduce` specification, this command does not report any instances of `master offset` in the logs of the `linuxptp` daemon.

## Troubleshooting common PTP Operator issues

Troubleshoot common problems with the PTP Operator by performing the following steps.

Prerequisites

- Install the OpenShift Container Platform CLI (`oc`).

- Log in as a user with `cluster-admin` privileges.

- Install the PTP Operator on a bare-metal cluster with hosts that support PTP.

Procedure

1. Check the Operator and operands are successfully deployed in the cluster for the configured nodes.

       $ oc get pods -n openshift-ptp -o wide

   Example output

       NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE
       linuxptp-daemon-lmvgn           3/3     Running   0          4d17h   10.1.196.24   compute-0.example.com
       linuxptp-daemon-qhfg7           3/3     Running   0          4d17h   10.1.196.25   compute-1.example.com
       ptp-operator-6b8dcbf7f4-zndk7   1/1     Running   0          5d7h    10.129.0.61   control-plane-1.example.com

   | When the PTP fast event bus is enabled, the number of ready `linuxptp-daemon` pods is `3/3`. If the PTP fast event bus is not enabled, `2/2` is displayed.

2. Check that supported hardware is found in the cluster.

       $ oc -n openshift-ptp get nodeptpdevices.ptp.openshift.io

   Example output

       NAME                                  AGE
       control-plane-0.example.com           10d
       control-plane-1.example.com           10d
       compute-0.example.com                 10d
       compute-1.example.com                 10d
       compute-2.example.com                 10d

3. Check the available PTP network interfaces for a node:

       $ oc -n openshift-ptp get nodeptpdevices.ptp.openshift.io <node_name> -o yaml

   where:

   <node_name>

     Specifies the node you want to query, for example, `compute-0.example.com`.

     Example output

         apiVersion: ptp.openshift.io/v1
         kind: NodePtpDevice
         metadata:
           creationTimestamp: "2021-09-14T16:52:33Z"
           generation: 1
           name: compute-0.example.com
           namespace: openshift-ptp
           resourceVersion: "177400"
           uid: 30413db0-4d8d-46da-9bef-737bacd548fd
         spec: {}
         status:
           devices:
           - name: eno1
           - name: eno2
           - name: eno3
           - name: eno4
           - name: enp5s0f0
           - name: enp5s0f1

4. Check that the PTP interface is successfully synchronized to the primary clock by accessing the `linuxptp-daemon` pod for the corresponding node.

   1. Get the name of the `linuxptp-daemon` pod and corresponding node you want to troubleshoot by running the following command:

          $ oc get pods -n openshift-ptp -o wide

      Example output

          NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE
          linuxptp-daemon-lmvgn           3/3     Running   0          4d17h   10.1.196.24   compute-0.example.com
          linuxptp-daemon-qhfg7           3/3     Running   0          4d17h   10.1.196.25   compute-1.example.com
          ptp-operator-6b8dcbf7f4-zndk7   1/1     Running   0          5d7h    10.129.0.61   control-plane-1.example.com

   2. Remote shell into the required `linuxptp-daemon` container:

          $ oc rsh -n openshift-ptp -c linuxptp-daemon-container <linux_daemon_container>

      where:

      <linux_daemon_container>

        is the container you want to diagnose, for example `linuxptp-daemon-lmvgn`.

   3. In the remote shell connection to the `linuxptp-daemon` container, use the PTP Management Client (`pmc`) tool to diagnose the network interface. Run the following `pmc` command to check the sync status of the PTP device, for example `ptp4l`.

          # pmc -u -f /var/run/ptp4l.0.config -b 0 'GET PORT_DATA_SET'

      Example output when the node is successfully synced to the primary clock

          sending: GET PORT_DATA_SET
              40a6b7.fffe.166ef0-1 seq 0 RESPONSE MANAGEMENT PORT_DATA_SET
                  portIdentity            40a6b7.fffe.166ef0-1
                  portState               SLAVE
                  logMinDelayReqInterval  -4
                  peerMeanPathDelay       0
                  logAnnounceInterval     -3
                  announceReceiptTimeout  3
                  logSyncInterval         -4
                  delayMechanism          1
                  logMinPdelayReqInterval -4
                  versionNumber           2

5. For GNSS-sourced grandmaster clocks, verify that the in-tree NIC ice driver is correct by running the following command, for example:

       $ oc rsh -n openshift-ptp -c linuxptp-daemon-container linuxptp-daemon-74m2g ethtool -i ens7f0

   Example output

       driver: ice
       version: 5.14.0-356.bz2232515.el9.x86_64
       firmware-version: 4.20 0x8001778b 1.3346.0

6. For GNSS-sourced grandmaster clocks, verify that the `linuxptp-daemon` container is receiving signal from the GNSS antenna. If the container is not receiving the GNSS signal, the `/dev/gnss0` file is not populated. To verify, run the following command:

       $ oc rsh -n openshift-ptp -c linuxptp-daemon-container linuxptp-daemon-jnz6r cat /dev/gnss0

   Example output

       $GNRMC,125223.00,A,4233.24463,N,07126.64561,W,0.000,,300823,,,A,V*0A
       $GNVTG,,T,,M,0.000,N,0.000,K,A*3D
       $GNGGA,125223.00,4233.24463,N,07126.64561,W,1,12,99.99,98.6,M,-33.1,M,,*7E
       $GNGSA,A,3,25,17,19,11,12,06,05,04,09,20,,,99.99,99.99,99.99,1*37
       $GPGSV,3,1,10,04,12,039,41,05,31,222,46,06,50,064,48,09,28,064,42,1*62

## Getting the DPLL firmware version for the CGU in an Intel 800 series NIC

You can get the digital phase-locked loop (DPLL) firmware version for the Clock Generation Unit (CGU) in an Intel 800 series NIC by opening a debug shell to the cluster node and querying the NIC hardware.

Prerequisites

- You have installed the OpenShift CLI (`oc`).

- You have logged in as a user with `cluster-admin` privileges.

- You have installed an Intel 800 series NIC in the cluster host.

- You have installed the PTP Operator on a bare-metal cluster with hosts that support PTP.

Procedure

1. Start a debug pod by running the following command:

       $ oc debug node/<node_name>

   where:

   <node_name>

     Is the node where you have installed the Intel 800 series NIC.

2. Check the CGU firmware version in the NIC by using the `devlink` tool and the bus and device name where the NIC is installed. For example, run the following command:

       sh-4.4# devlink dev info <bus_name>/<device_name> | grep cgu

   where:

   <bus_name>

     Is the bus where the NIC is installed. For example, `pci`.

   <device_name>

     Is the NIC device name. For example, `0000:51:00.0`.

   Example output

       cgu.id 36 (1)
       fw.cgu 8032.16973825.6021 (2)

   1 | CGU hardware revision number
   2 | The DPLL firmware version running in the CGU, where the DPLL firmware version is `6201`, and the DPLL model is `8032`. The string `16973825` is a shorthand representation of the binary version of the DPLL firmware version (`1.3.0.1`).

   | The firmware version has a leading nibble and 3 octets for each part of the version number. The number `16973825` in binary is `0001 0000 0011 0000 0000 0000 0001`. Use the binary value to decode the firmware version. For example:
   Table 11. DPLL firmware version
   Binary part | Decimal value
   --- | ---
   `0001` | 1
   `0000 0011` | 3
   `0000 0000` | 0
   `0000 0001` | 1

## Collecting PTP Operator data

You can use the `oc adm must-gather` command to collect information about your cluster, including features and objects associated with PTP Operator.

Prerequisites

- You have access to the cluster as a user with the `cluster-admin` role.

- You have installed the OpenShift CLI (`oc`).

- You have installed the PTP Operator.

Procedure

- To collect PTP Operator data with `must-gather`, you must specify the PTP Operator `must-gather` image.

      $ oc adm must-gather --image=registry.redhat.io/openshift4/ptp-must-gather-rhel8:v4.14