lock, which is most frequent during periods of darkness over the
propagation path, which occured roughly between 0800 UT and 1100 UT
during the experiment period. Excluding all data except that
collected during this period, the mean offset is -21.3 ms with
standard deviation in the range 29-31. The maximum offset is 59 ms
and the minimum is -118 ms.
In order to compute the discrepancy between the WWV and WWVB clocks,
it is necessary to subtract the total of the propagation delay plus
WWVB clock delay from the mean offsets computed above. Thus, the WWV
clock indications are -21.3 - 10 - 32 = -72.3 ms late (72.3 ms early)
with respect to the WWVB clock indications. Considering the large
standard deviations noted above, it is probably not worthwhile to
include this correction in the WWV clock indications.
On exceptional occasions excursions in offset over 300 ms relative to
the WWVB clock were observed. Close inspection of the data showed
that this was due to an extended period (a day or more) in which lock
RFC957 September 1985
Experiments in Network Clock Synchronization
was not achieved on any frequency. The master oscillator uses a
3.6-MHz crystal oscillator trimmed by a digital/analog converter and
register which is loaded by the microprocessor. The occasional
excursions in offset were apparently due to incorrect register values
as the result of noisy reception conditions and excessive intervals
between lock. On occasion the oscillator frequency was observed in
error over 4 ppm due to this cause, which could result in a
cumulative error of almost 400 ms per day if uncorrected.
4.3.4. On Handling Disruptions
The experiment period was intentionally selected to coincide with the
insertion of a leap second in the worldwide time broadcasts. The
intent was to examine the resulting behavior of the various radio
clocks and the synchronization algorithm when an additional second
was introduced at 2400 UT on 30 June.
As it turned out, radio reception conditions at the time of insertion
were quite poor on all WWV frequencies, the WWVB frequency and the
GOES frequency. Thus, all three clocks took varying periods up to
several hours to resynchonize and correct the indicated time. In
fact, the only time signals heard around the time of interest were
those from Canadian radio CHU, but the time code of the Canadian
broadcasts is incompatible with the of the US broadcasts.
As mentioned above, the WWVB clock was used as the master during the
experiment period. About two hours after insertion of the leap
second the clock resynchronized and all hosts in the experimental
network were corrected shortly afterwards. Since the magnitude of
the correction exceeded 128 ms, the correction was of a step nature,
but was not performed simultaneously in all hosts due to the
individual timing of the Hello messages. Thus, if timing-critical
network operations happened to take place during the correction
process, inconsistent timestamps could result.
The lesson drawn from this experience is quite clear. Accurate time
synchronization requires by its very nature long integration times,
so that epochal events which disrupt the process must be predicted in
advance and applied in all hosts independently. In principle, this
would not be hard to do and could even be integrated into the
operation of the step-correction procedure described earlier, perhaps
in the form of bits included in Hello messages which trigger a
one-second correction at the next rollover from 2400 to 0000 hours.
In order for such an out-of-band correction to be effective, advance
notice of the leap second must be available. At present, this
information is not available in the broadcast format and must be
RFC957 September 1985
Experiments in Network Clock Synchronization
obtained via the news media. In fact, there are spare bits in the
broadcast format that could be adapted for this purpose, but this
would require reprogramming both the transmitting and receiving
equipment. Nevertheless, this feature should be considered for future
systems.
4.4. Additional Experiments
A set of experiments was performed using two WIDEBAND/EISN gateways
equipped with WWVB radio clocks and connected to the ARPANET. These
experiments were designed to determine the limits of accuracy when
comparing these clocks via ARPANET paths. One of the gateways
(ISI-MCON-GW) is located at the Information Sciences Institute near
Los Angeles, while the other (LL-GW) is located at Lincoln
Laboratories near Boston. Both gateways consist of PDP11/44
computers running the EPOS operating system and clock-interface
boards with oscillators phase-locked to the WWVB clock.
The clock indications of the WIDEBAND/EISN gateways were compared
with the DCNet WWVB reference clock using ICMP Timestamp messages
[6], which record the individual timestamps with a precision of a
millisecond. This technique is not as accurate as the one described
in Section 3, since the protocol implementation involves the
user-process level, which can be subject to minor delays due to
process scheduling and interprocess-message queueing. However,
calibration measurements made over several of the links shown in
Figure 2 indicate that the measurement errors are dominated by the
individual link variations and not by the characteristics of the
measurement technique itself.
Measurements were made separately with each gateway by sending an
ICMP Timestamp Request message from the ARPANET address of DCN1 to
the ARPANET address of the gateway and computing the round-trip delay
and clock offset from the ICMP Timestamp Reply message. This process
was continued for 1000 message exchanges, which took about seven
minutes. Table 6 shows the statistics obtained with ISI-MCON-GW and
Table 7 those with LL-GW (all numbers are milliseconds).
RFC957 September 1985
Experiments in Network Clock Synchronization
ISI-MCON-GW Mean Dev Max Min
--------------------------------------------
Offset -16 40 126 -908
Delay 347 59 902 264
Table 6. ISI-MCON-GW Clock Statistics
LL-GW (a) Mean Dev Max Min
--------------------------------------------
Offset -23 15 32 -143
Delay 310 25 536 252
Table 7. LL-GW Clock Statistics
The smaller values of standard deviation and extreme for LL-GW are
probably due to the shorter ARPANET path involved. The confidence in
the mean offset can be estimated by dividing the standard deviation
by the square root of the number of samples (1000), which suggests
that the mean offsets are accurate to within a couple of miliseconds.
The mean offsets of the WIDEBAND/EISN clocks as a group relative to
the DCN1 clock may thus indicate a minor discrepancy in the setting
of the delay-compensation switches.
It is well known that ARPANET paths exhibit wide variations in
delays, with occasional delays reaching surprising values up to many
seconds. In order to improve the estimates a few samples were
removed from both the offset and delay data, including all those with
magnitude greater than one second.
The above experiments involve a burst of activity over a relatively
short time during which the ratio of the measurement traffic to other
network traffic may be nontrivial. Another experiment with LL-GW was
designed with intervals of ten seconds between ICMP messages and
operated over a period of about three hours. The results are shown
in Table 8.
LL-GW (b) Mean Dev Max Min
--------------------------------------------
Offset -16 93 990 -874
Delay 371 108 977 240
Table 8. LL-GW Clock Statistics
RFC957 September 1985
Experiments in Network Clock Synchronization
Note that the standard deviations and extrema are higher than in the
previous experiments, but the mean offset is about the same.
The results of these experiments suggest that time synchronization
via ARPANET paths can yield accuracies to the order of a few
milliseconds, but only if relatively large numbers of samples are
available. The number of samples can be reduced and the accuracy
improved by using the techniques of Section 3 modified for ICMP
Timestamp messages and the longer, more noisy paths involved.
5. Summary and Conclusions
The experiments described above were designed to verify the correct
operation of the DCnet time-synchronization algorithms and protocols
under a variety of scenarios, including the use of line-frequency
clocks, three types of radio clocks and various types of
interprocessor links. They involved the collection and processing of
many megabytes of data collected over a ten-day period that included
the insertion of a leap second in the standard NBS time scale. Among
the lessons learned were the following:
1. The algorithms and protocols operate as designed, yielding
accuracies throughout the experimental net in the order of a
few milliseconds to a few tens of milliseconds, depending on
the topology and link type.
2. Glitches due to congestion, rebooted hosts and link failures
are acceptably low, even in the face of massive congestion
resulting from inappropriate host implementations elsewhere in
the Internet.
3. A synchronization scenario where the clocks in all hosts are
locked to the line frequency and corrections are broadcast
from a central time standard will work only if all hosts are
on the same power grid, which is unlikely in the present
Internet configuration, but may be appropriate for some
applications.
4. In spite of the eastern power grid wandering over as much as
six seconds in a day, it is possible to achieve accuracies in
the 30-ms range using line-frequency interface clocks and
corrections broadcast on the local net.
5. Radio clocks can vary widely in accuracy depending on signal
reception conditions. Absolute time can be determined to
within a couple of milliseconds using WWVB and GOES radio
clocks, but only if they are calibrated using an independent
RFC957 September 1985
Experiments in Network Clock Synchronization
standard such as a portable clock. The inexpensive WWV clocks
perform surprisingly well most of the time, but can be in
error up to a significant fraction of a second under some
conditions.
6. Adjustments in the time scale due to leap seconds must be
anticipated before they occur. The synchronization protocol
must include a mechanism to broadcast an adjustment in advance
of its occurance, so that it can be incorporated in each host
simultaneously. There is a need to incorporate advance notice
of leap seconds in the broadcast time code.
7. Time synchronization via ARPANET paths can yield accuracies in
the order of a few milliseconds, but only if relatively large
numbers of samples are available. Further work is needed to
develop efficient protocols capable of similar accuracies but
using smaller numbers of samples.
6. References
1. Lindsay, W.C., and A.V. Kantak. Network Synchronization of
Random Signals. IEEE Trans. Comm. COM-28, 8 (August 1980),
1260-1266.
2. Mills, D.L. Time Synchronization in DCNET Hosts. DARPA Internet
Project Report IEN-173, COMSAT Laboratories, February 1981.
3. Mills, D.L. DCNET Internet Clock Service. DARPA Network Working
Group Report RFC-778, COMSAT Laboratories, April 1981.
4. Mills, D.L. Internet Delay Experiments. DARPA Network Working
Group Report RFC-889, M/A-COM Linkabit, December 1983.
5. Mills, D.L. DCN Local-Network Protocols. DARPA Network Working
Group Report RFC-891, M/A-COM Linkabit, December 1983.
6. Postel, J. Internet Control Message Protocol. DARPA Network
Working Group Report RFC-792, USC Information Sciences Institute,
September 1981.
7. Postel, J. Time Protocol. DARPA Network Working Group Report
RFC-868, USC Information Sciences Institute, May 1983.
8. Postel, J. Daytime Protocol. DARPA Network Working Group Report
RFC-867, USC Information Sciences Institute, May 1983.
RFC957 September 1985
Experiments in Network Clock Synchronization
9. Su, Z. A Specification of the Internet Protocol (IP) Timestamp
Option. DARPA Network Working Group Report RFC-781. SRI
International, May 1981.
10. Marzullo, K., and S. Owicki. Maintaining the Time in a
Distributed System. ACM Operating Systems Review 19, 3 (July
1985), 44-54.
11. Mills, D.L. Algorithms for Synchronizing Network Clocks. DARPA
Network Working Group Report RFC-956, M/A-COM Linkabit, September
1985.
12. Mills, D.L. Network Time Protocol (NTP). DARPA Network Working
Group Report RFC-958, M/A-COM Linkabit, September 1985.