PTP Quickstart Guide
Overview
There are many configurations for a PTP network, this quick start guide aims to cover the basics by using Ubuntu 18.04 as an example. It provides configuration settings for a commercial PTP grandmaster clock and also provides directions on setting up a Linux computer (Ubuntu 18.04) to function as a PTP grandmaster.
The linuxptp project provides a suite of PTP tools that can be used to serve as a PTP master clock for a local network of sensors.
Assumptions
Command line Linux knowledge (e.g., package management, command line familiarity, etc.).
Ethernet interfaces that support hardware timestamping.
Ubuntu 18.04 is assumed for this tutorial, but any modern distribution should suffice.
Knowledge of systemd service configuration and management.
Familiarity with Linux permissions.
Physical Network Setup
Ensure the Ouster sensor is connected to the PTP master clock with at most one network switch. Ideally the sensor should be connected directly to the PTP grandmaster. Alternatively, a simple layer-2 gigabit Ethernet switch will suffice. Multiple switches are not recommended and will add unnecessary jitter.
Third Party Grandmaster Clock
A dedicated grandmaster clock should be used for the highest absolute accuracy often with a GPS receiver.
It must be configured with the following parameters which match the linuxptp client defaults:
Transport:
UDP IPv4Delay Mechanism:
E2ESync Mode:
Two-StepAnnounce Interval:
1- sent every 2 secondsSync Interval:
0- sent every 1 secondDelay Request Interval:
0- sent every 1 second
For more settings, review the port_data_set field returned from the sensor’s HTTP /time/ptp interface.
Linux PTP Grandmaster Clock
An alternative to an external grandmaster PTP clock is to run a local Linux PTP master clock if accuracy allows. This is often implemented on a vehicle computer that interfaces directly with the lidar sensors.
This section outlines how to configure a master clock.
Example Network Setup
This section assumes the following network setup as it has elements of a local master clock and the option for an upstream PTP time source.
+-------------------------------------+
| Ubuntu 18.04 System |
| * 2x Intel i210 Ethernet Interfaces |
| * Linux PTP service |
| |
| eno1 eno2 |
+-------+---------------------+-------+
| |
+-------+-------+ +--------+------+
| Trimble GM100 | | + +
| GPS -> PTP | | Ouster OS1 | |
| grandmaster | | | |
| (optional) | | | |
+---------------+ +---------------- |
+--------------- +
The focus is on configuring the Linux PTP service to serve a common clock to all the downstream Ouster OS1 sensors using the Linux system time from the Ubuntu host machine.
Optionally, a grandmaster clock can be added to discipline the system time of the Linux host.
Installing Necessary Packages
Several packages are needed for PTP functionality and verification:
linuxptp- Linux PTP package with the following components:ptp4ldaemon to manage hardware and participate as a PTP nodephc2systo synchronize the Ethernet controller’s hardware clock to the Linux system clock or shared memory regionpmcto query the PTP nodes on the network.
chrony- A NTP and PTP time synchronization daemon. It can be configured to listen to both NTP time sources via the Internet and a PTP master clock such as one provided by a GPS with PTP support. This will validate the time configuration makes sense given multiple time sources.ethtool- A tool to query the hardware and driver capabilities of a given Ethernet interface.
$ sudo apt update
...
Reading package lists... Done
Building dependency tree
Reading state information... Done
$ sudo apt install linuxptp chrony ethtool
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
chrony ethtool linuxptp
0 upgraded, 3 newly installed, 0 to remove and 29 not upgraded.
Need to get 430 kB of archives.
After this operation, 1,319 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 ethtool amd64 1:4.15-0ubuntu1 [114 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 linuxptp amd64 1.8-1 [112 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 chrony amd64 3.2-4ubuntu4.2 [203 kB]
Fetched 430 kB in 1s (495 kB/s)
Selecting previously unselected package ethtool.
(Reading database ... 117835 files and directories currently installed.)
Preparing to unpack .../ethtool_1%3a4.15-0ubuntu1_amd64.deb ...
Unpacking ethtool (1:4.15-0ubuntu1) ...
Selecting previously unselected package linuxptp.
Preparing to unpack .../linuxptp_1.8-1_amd64.deb ...
Unpacking linuxptp (1.8-1) ...
Selecting previously unselected package chrony.
Preparing to unpack .../chrony_3.2-4ubuntu4.2_amd64.deb ...
Unpacking chrony (3.2-4ubuntu4.2) ...
Setting up linuxptp (1.8-1) ...
Processing triggers for ureadahead (0.100.0-20) ...
ureadahead will be reprofiled on next reboot
Setting up chrony (3.2-4ubuntu4.2) ...
Processing triggers for systemd (237-3ubuntu10.13) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up ethtool (1:4.15-0ubuntu1) ...
Ethernet Hardware Timestamp Verification
- Identify the Ethernet interface to be used on the client (Linux) machine,
e.g., eno1. Run the ethtool utility and query this network interface for supported capabilities.
Output of ethtool -T for a functioning Intel i210 Ethernet interface:
$ sudo ethtool -T eno1
Time stamping parameters for eno1:
Capabilities:
hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
off (HWTSTAMP_TX_OFF)
on (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
none (HWTSTAMP_FILTER_NONE)
all (HWTSTAMP_FILTER_ALL)
Configurations
Configuring ptp4l for Multiple Ports
On a Linux system with multiple Ethernet ports (i.e. Intel i210) /etc/linuxptp/ptp4l.conf needs to be configured to support all of them.
boundary_clock_jbod 1
[eno1]
[eno2]
Note
Add the above required modification at the end of the existing file. Deleting or editing the default settings section of the ptp4l.conf file will result in an error.
The default systemd service file for Ubuntu 18.04 attempts to use the eth0 address on the command line. Override systemd service file so that the configuration file is used instead of hard coded in the service file.
Create a systemd drop-in directory to override the system service file:
$ sudo mkdir -p /etc/systemd/system/ptp4l.service.d
Create a file at /etc/systemd/system/ptp4l.service.d/override.conf with
the following contents:
[Service]
ExecStart=
ExecStart=/usr/sbin/ptp4l -f /etc/linuxptp/ptp4l.conf
Restart the ptp4l service so the change takes effect:
$ sudo systemctl daemon-reload
$ sudo systemctl restart ptp4l
$ sudo systemctl status ptp4l
* ptp4l.service - Precision Time Protocol (PTP) service
Loaded: loaded (/lib/systemd/system/ptp4l.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/ptp4l.service.d
└─override.conf
Active: active (running) since Wed 2019-03-13 14:38:57 PDT; 3s ago
Docs: man:ptp4l
Main PID: 25783 (ptp4l)
Tasks: 1 (limit: 4915)
CGroup: /system.slice/ptp4l.service
└─25783 /usr/sbin/ptp4l -f /etc/linuxptp/ptp4l.conf
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.756] port 1: INITIALIZING to LISTENING on INITIALIZE
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.756] driver changed our HWTSTAMP options
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.756] tx_type 1 not 1
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.756] rx_filter 1 not 12
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.756] port 2: INITIALIZING to LISTENING on INITIALIZE
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.757] port 0: INITIALIZING to LISTENING on INITIALIZE
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.757] port 1: link up
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.757] port 2: link down
Mar 13 14:38:57 leadlizard ptp4l[25783]: [590188.757] port 2: LISTENING to FAULTY on
FAULT_DETECTED (FT_UNSPECIFIED)
Mar 13 14:38:58 leadlizard ptp4l[25783]: [590189.360] port 1: new foreign master 001747.fffe.700038-1
The above systemctl status ptp4l console output shows systemd correctly reading the override file created earlier before starting several seconds after the restart command.
The log output shows that a grandmaster clock has been discovered on port 1 (eno1) and port 2 (eno2) is currently disconnected and in the faulty state as expected. In the test network a Trimble Thunderbolt PTP GM100 Grandmaster Clock is attached on eno1.
Logs can be monitored (i.e. followed) like so:
$ journalctl -f -u ptp4l
-- Logs begin at Fri 2018-11-30 06:40:50 PST. --
Mar 13 14:51:37 leadlizard ptp4l[25783]: [590948.224] master offset -17 s2 freq -25963 path delay 14183
Mar 13 14:51:38 leadlizard ptp4l[25783]: [590949.224] master offset -13 s2 freq -25964 path delay 14183
Mar 13 14:51:39 leadlizard ptp4l[25783]: [590950.225] master offset 35 s2 freq -25920 path delay 14192
Mar 13 14:51:40 leadlizard ptp4l[25783]: [590951.225] master offset -59 s2 freq -26003 path delay 14201
Mar 13 14:51:41 leadlizard ptp4l[25783]: [590952.225] master offset -24 s2 freq -25986 path delay 14201
Mar 13 14:51:42 leadlizard ptp4l[25783]: [590953.225] master offset -39 s2 freq -26008 path delay 14201
Mar 13 14:51:43 leadlizard ptp4l[25783]: [590954.225] master offset 53 s2 freq -25928 path delay 14201
Mar 13 14:51:44 leadlizard ptp4l[25783]: [590955.226] master offset -85 s2 freq -26050 path delay 14207
Mar 13 14:51:45 leadlizard ptp4l[25783]: [590956.226] master offset 127 s2 freq -25863 path delay 14207
Mar 13 14:51:46 leadlizard ptp4l[25783]: [590957.226] master offset 9 s2 freq -25943 path delay 14208
Mar 13 14:51:47 leadlizard ptp4l[25783]: [590958.226] master offset -23 s2 freq -25973 path delay 14208
Mar 13 14:51:48 leadlizard ptp4l[25783]: [590959.226] master offset -61 s2 freq -26018 path delay 14190
Mar 13 14:51:49 leadlizard ptp4l[25783]: [590960.226] master offset 69 s2 freq -25906 path delay 14190
Mar 13 14:51:50 leadlizard ptp4l[25783]: [590961.226] master offset -73 s2 freq -26027 path delay 14202
Mar 13 14:51:51 leadlizard ptp4l[25783]: [590962.226] master offset 19 s2 freq -25957 path delay 14202
Mar 13 14:51:52 leadlizard ptp4l[25783]: [590963.226] master offset 147 s2 freq -25823 path delay 14202
...
Configuring ptp4l as a Local Master Clock
The IEEE-1588 Best Master Clock Algorithm (BMCA) will select a grandmaster clock based on a number of masters. In most networks there should be only a single master. In the example network the Ubuntu machine will be configured with a non-default clockClass so its operation qualifies it to win the BMCA.
Replace the default value with a lower clock class (higher priority) and restart linuxptp. Edit /etc/linuxptp/ptp4l.conf and comment out the default clockClass value and insert a line setting it 128.
#clockClass 248
clockClass 128
Restart ptp4l so the configuration change takes effect.
$ sudo systemctl restart ptp4l
This will configure ptp4l to advertise a master clock on eno2 as a clock that will win the BMCA for an Ouster OS1 sensor.
However, the ptp4l service is only advertising the Ethernet controller’s PTP hardware clock, not the Linux system time as is often expected.
Configuring ``phc2sys`` to Synchronize the System Time to the PTP Clock
To synchronize the Linux system time to the PTP hardware clock the phc2sys utility needs to be run. The following configuration will tell phc2sys to take the Linux CLOCK_REALTIME and write that time to the PTP hardware clock in the Ethernet controller for eno2. These interfaces are then connected to PTP slaves such as Ouster OS1 sensors.
Create a systemd drop-in directory to override the system service file:
$ sudo mkdir -p /etc/systemd/system/phc2sys.service.d
Create a file at /etc/systemd/system/phc2sys.service.d/override.conf with the following contents:
[Service]
ExecStart=
ExecStart=/usr/sbin/phc2sys -w -s CLOCK_REALTIME -c eno2
Note
If multiple interfaces need to be synchronized from CLOCK_REALTIME then multiple instances of the phc2sys service need to be run as it only accepts a single slave (i.e. -c) argument.
Restart the phc2sys service so the change takes effect:
$ sudo systemctl daemon-reload
$ sudo systemctl restart phc2sys
$ sudo systemctl status phc2sys
Configuring Chrony to Set System Clock Using PTP
An upstream PTP grandmaster clock (e.g., a GPS disciplined PTP clock) can be used to set the system time if precise absolute time is needed for sensor data.
Chrony is a Linux time service that can read from NTP and PTP and set the Linux system time using the most accurate source available. With a properly functioning PTP grandmaster the PTP time source will be selected and the error from the public time servers can be reviewed.
The following phc2shm service will synchronize the time from eno1 (where the external grandmaster is attached) to the system clock.
Create a file named /etc/systemd/system/phc2shm.service with the following
contents:
# /etc/systemd/system/phc2shm.service
[Unit]
Description=Synchronize PTP hardware clock (PHC) to NTP SHM
Documentation=man:phc2sys
After=ntpdate.service
Requires=ptp4l.service
After=ptp4l.service
[Service]
Type=simple
ExecStart=/usr/sbin/phc2sys -s eno1 -E ntpshm -w
[Install]
WantedBy=multi-user.target
Then start the newly created service and check that it started.
$ sudo systemctl start phc2shm
$ sudo systemctl status phc2shm
Add the PTP time source to the chrony configuration which will read the shared memory region managed by the phc2shm service created above.
Append the following to the /etc/chrony/chrony.conf file:
refclock SHM 0 poll 1 refid ptp
Restart chrony so the updated configuration file takes effect:
$ sudo systemctl restart chrony
After waiting a minute for the clock to synchronize, review the chrony client timing accuracy:
$ chronyc tracking
Reference ID : 70747000 (ptp)
Stratum : 1
Ref time (UTC) : Thu Mar 14 02:22:58 2019
System time : 0.000000298 seconds slow of NTP time
Last offset : -0.000000579 seconds
RMS offset : 0.001319735 seconds
Frequency : 0.502 ppm slow
Residual freq : -0.028 ppm
Skew : 0.577 ppm
Root delay : 0.000000001 seconds
Root dispersion : 0.000003448 seconds
Update interval : 2.0 seconds
Leap status : Normal
$ chronyc sources -v
210 Number of sources = 9
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
#* ptp 0 1 377 1 +27ns[ +34ns] +/- 932ns
^- chilipepper.canonical.com 2 6 377 61 -482us[ -482us] +/- 99ms
^- pugot.canonical.com 2 6 377 62 -498us[ -498us] +/- 112ms
^- golem.canonical.com 2 6 337 59 -467us[ -468us] +/- 95ms
^- alphyn.canonical.com 2 6 377 58 +957us[ +957us] +/- 95ms
^- legacy13.chi1.ntfo.org 3 6 377 62 -10ms[ -10ms] +/- 178ms
^- tesla.selinc.com 2 6 377 128 +429us[ +514us] +/- 42ms
^- io.crash-override.org 2 6 377 59 +441us[ +441us] +/- 58ms
^- hadb2.smatwebdesign.com 3 6 377 58 +1364us[+1364us] +/- 99ms
Note that the Reference ID matches the ptp reference ID from the chrony.conf file and that the sources output shows the ptp reference ID as selected (signified by the * state in the second column). Additionally, the NTP time sources show a small relative error to the high accuracy PTP time source.
In this case the PTP grandmaster is properly functioning.
If this error is large, chrony will select the NTP time sources and mark the PTP time source as invalid. This typically signifies that something is mis-configured with the PTP grandmaster upstream of this device or the linuxptp configuration.
Verifying Operation
If the PTP grandmaster was just set up and configured, it’s recommended to power cycle the sensor. The sensor will then jump to the correct time instead of slowly easing in the time adjustment which will take time if the grandmaster initially set the sensor to the wrong time.
Sensor PTP Sync Verification
The sensor can be queried for the state of its local PTP service through the GET /api/v1/time/ptp request.
JSON response fields to check:
parent_data_set.grandmaster_identityshould list the identity of the local grandmasterport_data_set.port_stateshould beSLAVEtime_status_np.gm_presentshould betruetime_status_np.master_offsetwhich is given in nanoseconds, should be less than250000. This equates to 250 microseconds.
PTP Example JSON Response
{
"profile": "default",
"parent_data_set":
{
"grandmaster_identity": "001747.fffe.700038",
"parent_port_identity": "ac1f6b.fffe.1db84e-2",
"parent_stats": 0,
"gm_clock_class": 6,
"observed_parent_clock_phase_change_rate": 2147483647,
"gm_clock_accuracy": 33,
"gm_offset_scaled_log_variance": 65535,
"grandmaster_priority1": 128,
"grandmaster_priority2": 128,
"observed_parent_offset_scaled_log_variance": 65535
},
"current_data_set":
{
"steps_removed": 1,
"offset_from_master": 61355,
"mean_path_delay": 117977.0
},
"port_data_set":
{
"port_state": "SLAVE",
"peer_mean_path_delay": 0,
"log_min_delay_req_interval": 0,
"port_identity": "bc0fa7.fffe.c48254-1",
"log_sync_interval": 0,
"log_announce_interval": 1,
"delay_mechanism": 1,
"log_min_pdelay_req_interval": 0,
"announce_receipt_timeout": 3,
"version_number": 2
},
"time_status_np":
{
"gm_time_base_indicator": 0,
"gm_identity": "001747.fffe.700038",
"cumulative_scaled_rate_offset": 0,
"scaled_last_gm_phase_change": 0,
"ingress_time": 0,
"master_offset": 61355,
"last_gm_phase_change": "0x0000'0000000000000000.0000",
"gm_present": true
},
"time_properties_data_set":
{
"frequency_traceable": 0,
"leap61": 0,
"time_traceable": 0,
"current_utc_offset": 37,
"leap59": 0,
"current_utc_offset_valid": 0,
"time_source": 160,
"ptp_timescale": 1
}
}
LinuxPTP PMC Tool
The sensor will respond to PTP management messages. The linuxptp pmc (see man pmc) utility can be used to query all PTP devices on the local network.
On the Linux host for the pmc utility to communicate with then run the following command:
$ sudo pmc 'get PARENT_DATA_SET' 'get CURRENT_DATA_SET' 'get PORT_DATA_SET' 'get TIME_STATUS_NP' -i eno2
sending: GET PARENT_DATA_SET
sending: GET CURRENT_DATA_SET
sending: GET PORT_DATA_SET
sending: GET TIME_STATUS_NP
bc0fa7.fffe.c48254-1 seq 0 RESPONSE MANAGEMENT PARENT_DATA_SET
parentPortIdentity ac1f6b.fffe.1db84e-2
parentStats 0
observedParentOffsetScaledLogVariance 0xffff
observedParentClockPhaseChangeRate 0x7fffffff
grandmasterPriority1 128
gm.ClockClass 6
gm.ClockAccuracy 0x21
gm.OffsetScaledLogVariance 0x4e5d
grandmasterPriority2 128
grandmasterIdentity 001747.fffe.700038
bc0fa7.fffe.c48254-1 seq 1 RESPONSE MANAGEMENT CURRENT_DATA_SET
stepsRemoved 2
offsetFromMaster 61355.0
meanPathDelay 117977.0
bc0fa7.fffe.c48254-1 seq 2 RESPONSE MANAGEMENT PORT_DATA_SET
portIdentity bc0fa7.fffe.c48254-1
portState SLAVE
logMinDelayReqInterval 0
peerMeanPathDelay 0
logAnnounceInterval 1
announceReceiptTimeout 3
logSyncInterval 0
delayMechanism 1
logMinPdelayReqInterval 0
versionNumber 2
bc0fa7.fffe.c48254-1 seq 3 RESPONSE MANAGEMENT TIME_STATUS_NP
master_offset 61355
ingress_time 0
cumulativeScaledRateOffset +0.000000000
scaledLastGmPhaseChange 0
gmTimeBaseIndicator 0
lastGmPhaseChange 0x0000'0000000000000000.0000
gmPresent true
gmIdentity 001747.fffe.700038
Tested Grandmaster Clocks
- Trimble Thunderbolt PTP GM100 Grandmaster Clock
Firmware version:
20161111-0.1.4.0, November 11 2016 15:58:25PTP configuration:
> get ptp eth0 Enabled : Yes Clock ID : 001747.fffe.700038-1 Profile : 1588 Domain number : 0 Transport protocol : IPV4 IP Mode : Multicast Delay Mechanism : E2E Sync Mode : Two-Step Clock Class : 6 Priority 1 : 128 Priority 2 : 128 Multicast TTL : 0 Sync interval : 0 Del Req interval : 0 Ann. interval : 1 Ann. receipt timeout : 3
- Ubuntu 18.04 + Linux PTP as a master clock
Intel i210 Ethernet interface
PCI hardware identifiers:
8086:1533 (rev 03)
Ubuntu 18.04 kernel package:
linux-image-4.18.0-16-genericUbuntu 18.04 linuxptp package:
linuxptp-1.8-1