The Industrial Ethernet Book - Articles: IEEE 1588 and the need for Ethernet Clock Synchronisation

Technical Article: IEEE 1588 and the need for Ethernet Clock Synchronisation

Jump to Articles from:

Standard Ethernet was not designed for the rigorous operating conditions required in industrial automation. A primary challenge therefore has been how to coordinate all devices in a network, particularly in high end motion control applications where microsecond synchronisation, or better, is needed. One way to meet this challenge is with the IEEE 1588 Precision Time Protocol which synchronizes the local clock on each device with a grand master system clock. The capability of this approach is heavily dependent on a variety of factors.
By Alexander E Tan

Historically, time on an Ethernet network has been coordinated using the Network Time Protocol (NTP). This is sufficient for general clock updates, but is not precise enough for the type of time information needed in industrial automation, test and measurement, telecommunication, and similar demanding applications. NTP-based solutions offer, at best, tens of milliseconds synchronisation times (Neagoe, 2006), and are therefore unsuitable for high precision applications. This has led to the development of additional protocols to meet more challenging environments (Felser, 2005). One is called the IEEE 1588 Precision Time Protocol (PTP).

At its core, IEEE 1588 PTP uses an exchange of specially designed packets to provide time information from two connected nodes to calculate the difference in the time and frequency between the two clocks. The protocol also provides for a continuous process to adjust these clocks to stay in sync.

The IEEE 1588 protocol can be implemented purely in software, but for best results a hardware based solution is required. The primary advantage of a hardware solution is that specialised IEEE 1588 PTP components allow the packet timestamp to be taken as close to the wire as possible. The further from the wire, the more non-deterministic the jitter. The fundamental precision of the protocol is based on the precision of the timestamp. Once uncertainty is introduced, it can not be removed. Also, for the best possible performance, the entire real time network needs to be carefully designed. In addition to precise time stamps, the frequency offset of the slave node to the master needs to be compensated. This can be done by adjusting the IEEE 1588 slave clock, so accuracy is primarily limited by how finely the slave clock can be tuned. More error can be introduced from frequency drift between the clocks. The highest precision solutions benefit from using specialised oscillators that offer very high stability. Uncertainty in the packet delay through the network that may be introduced by switches or asymmetries in the path between the master clock and the adjusted node will also impact the overall system synchronisation. This can be best addressed by proper choice of components.

Software Implementation
The basic hardware architecture of an Ethernet device can be simplified down to a microcontroller or processor, a 10/100 Mbps Ethernet physical layer device (PHY) that receives the signals from an RJ-45 connector, and an Ethernet media access controller (MAC) that takes the signals from the PHY, validates their content, and converts them into digital signals for the microcontroller to use. The MAC is connected to the PHY through the media independent interface (MII) or the reduced media independent interface (RMII). The RMII is similar to the MII, but has fewer pins and runs at twice the rate. The basic communications software architecture has a MAC driver that controls the data exchange with the MAC and sets up and monitors the PHY. The MAC driver communicates with the PHY using a serial path through the MDIO, a signal on the MII or RMII interface. Above the MAC driver are the various protocol stacks that control communication functions such as IP and UDP. This architecture is represented by the blue components in the accompanying figure 1.

Figure 1: Basic Functional Blocks for an IEEE 1588 PTP Enabled System

There are several basic functions required to implement IEEE 1588 in an Ethernet-enabled device. The general operation of the protocol, recognizing and responding to packets and calculating adjustment to the local time, is handled in the IEEE 1588 PTP stack that sits above the other communication stacks in software. For IEEE 1588 PTP to work, the time that the packets containing the PTP information arrive and are sent needs to be accurately recorded. This process is handled in the IEEE 1588 timestamping unit.

Once the correct clock adjustments are calculated, the local time needs to be maintained in the IEEE 1588 clock. Another feature that is common in IEEE 1588 implementations - though not a strict requirement - is the ability to generate triggers and capture events. A trigger is used to send a signal at a particular point in time, and an event capture takes place when the exact time of an event is recorded. These features allow actions to occur with respect to the master clock for the network. They are represented by red components in figure 1. In general terms, the overall precision of an IEEE 1588 PTP system depends on the time critical packet timestamping, clock update, trigger, and event capture functions.

The most straightforward path to implementing IEEE 1588 PTP is to handle all of these additional functions as additional software. Figure 2 illustrates the division of these functions for a software-based system. Due to the nondeterministic nature of software, however, there are limits to how precise this can be. Uncompensated delays will be introduced when the time is recorded for outgoing and incoming packets, when the local clock is updated to the master time as the result of a synchronisation operation, when triggers are sent out, and when the time of events are recorded. Depending on whether or not a realtime operating system is being used, additional delays can come during context switching to handle interrupts and during high processing loads. The most critical function with respect to precision timing is to record the timestamp of incoming and outgoing packets.

In essence, every step in the process after the packet is received from the wire is an opportunity to add non deterministic delays and introduce errors to the quality of the time adjustments. Recording a time stamp close to the wire will result in the best possible node synchronisation. In a software implementation, the packet arrival and departure time is recorded at the processor, the farthest point from the wire, by software and thus has a higher level of nondeterministic delay built into the system. Software based systems will often have a best possible point to point precision in the order of hundreds of mS.

Figure 2: IEEE 1588 PTP Implemented in Software

Hardware implementation
Many industrial applications need precision of several orders of magnitude better than that delivered by a software solution. One method to get the timestamping unit closer to the wire is to dedicate an FPGA to handle the time critical IEEE 1588 PTP actions. Given the flexibility inherent in an FPGA, exact implementations vary, but a common choice has the FPGA placed outside of the normal communication path. By snooping on the PTP packets as they are sent between the MAC and the PHY, the send and received times as seen at the MII are recorded. This removes any nondeterministic behavior from the microcontroller and the MAC. An FPGA implementation can offer significantly better synchronisation than software depending on the choice of FPGA, the frequency that the FPGA is run at for the timestamp operations, and the details of the specific implementations. However, this approach requires the most effort on the part of the system designer as the entire IEEE 1588 PTP function has to be custom developed and adds additional component expenses. Figure 3.

Figure 3: IEEE 1588 PTP implemented with an FPGA

Chip-based IEEE 1588
Microcontroller vendors have been introducing microcontrollers with specific IEEE 1588 PTP hardware support to offer a simpler path to hardware based synchronisation. The precision improvement offered by these dedicated microcontrollers depends on the frequency that the timestamping occurs and the internal microarchitecture. In general, an IEEE 1588 PTP microcontroller has the Ethernet MAC incorporated into the microcontroller and has hardware support to allow timestamping of the PTP packets when they enter the MAC. Another common feature is to have specific support for local clock adjustments provided either in hardware or using microcode to reduce any nondeterministic behavior. Microcontrollers with IEEE 1588 PTP support are available now. Specialised microcontrollers require that existing designs be ported to the new microcontrollers, which can add significant cost and time to product development and offer limited choices.

Figure 4: IEEE 1588 Implemented in a Microcontroller

Both FPGA and specialised microcontroller solutions record the timestamp of the packet after it passes through the PHY. The only way to get to the highest level of synchronisation is to remove any source of nondeterministic delay from the PHY as well. This can be done by recording the timestamp as the packet enters the PHY. The next generation of industrial Ethernet PHYs will deliver this by providing hardware support for IEEE 1588 PTP functions in addition to 10/100 Ethernet. An example of this architecture is given in figure 4. The time critical IEEE 1588 time stamping, synchronized local clock, synchronized triggers and event capture is handled right at the wire. A significant benefit is that this configuration allows a synchronous clock output to be generated from the PHY that can drive additional devices and components to also be synchronized using the IEEE PTP 1588 protocol. The high level of precision is attained by using a PLL based clock output that is derived from the local IEEE 1588 PTP clock. An additionally advantage is that it allows the designer to have their own choice of microcontroller, requires no additional components, and takes up the same amount of board space as a non IEEE 1588 PTP enabled device. Point to point synchronisation in this scenario can be as low single digit nanoseconds, allowing the highest possible synchronisation across a network. Figure 5.

Figure 5: IEEE 1588 Implemented in the PHY

Additional steps can be taken to improve synchronisation. Temperature controlled oscillators allow for much more stable reference clocks, but at considerably higher prices. Another approach is to set a synchronous network to eliminate the drift. This can eliminate one source of imprecision due to drift between clocks. A 10/100 Ethernet synchronous network requires a PHY that can support synchronous mode operation where the reference clock and transmit data come from the clock recovered off of the receive signal and work best in strongly hierarchical networks. The latter requirement derives from the master/slave nature of synchronous operation. The PHY that sends the signal has to be closer on the network to the originating network clock and every node has to properly support synchronous operation for a synchronous network to be set up. In a synchronous network set up in this manner, IEEE 1588 PTP is still required to synchronize the time, but the effects of drift between clocks is eliminated.

Realizing High-Precision Clock Synchronisation The overall precision of a clock synchronized system is dependant on many factors. Once the question of communication protocol is resolved, the network topology has a strong influence on the level of precision possible. For instance, different topologies introduce different amounts of latency and synchronisation jitter. More flexible, dynamic networks may preclude the use of certain approaches, such as the hierarchical synchronous network described previously. Some industrial and communication protocols are targeted at specific topologies and help to improve the synchronisation and performance of certain network topologies. Finally, the choice of equipment is key to meeting clock synchronisation goals, as different switches, hugs, routers, gateways, and end nodes perform in a wide range for latency and introduced nondeterministic error. These sources of uncertainty are explored in more detail in figure 6.

Source of Uncertainty	Impact on Clock Synchronisation	Potential Mitigation
Timestamp	Inaccuracies in the timestamp reduce the precision of the fundamental clock adjustment calculations.	Moving the timestamp unit closer to the wire reduces uncertainty. Finer resolution clocking improves the timestamp precision. Deterministic design provides for the cleanest environment to take the timestamp.
Changes in the Frequency Offset Relative to Master	The frequency offset between the master and the slave is corrected through the PTP calculations.As this drift occurs, adjustments to the local clock can temporarily reduce the synchronisation between clocks.	Fine grained adjustments to local clock reduce uncertainty.
Frequency Stability to Local Clock	The synchronisation of the local clock to the master clock is based on the quality of the reference clock. The better the quality of the reference clock, the less drift is experienced and the more accurate the clock remains between synch updates.	OCXO/TCXO oscillators are available to reduce drift over temperature.
Latency Uncertainty in Packet Delay Over Network	Nondeterministic latency that is introduced at any stage in the network will increase the error in the time compensation calculation.	Proper network component and network topology selection reduces latency.
Asymmetric Data Path reduces Precision of IEEE 1588 protocol	The IEEE 1588 PTP protocol assumes symmetry between the transmit and receive paths in its calculations.	Network design and cable choices need to maximised the symmetry between transmit and receive paths.
Latency Delay in Servicing IEEE 1588 Packets	Long delays cause the synch attempt to fail. Servicing delays that impact the accuracy of the timestamp reduce the system synchronisation precision.	Reduce the load associated with clock synchronisation operations to improve response to PTP packets.

Figure 6: Sources of System Synchronisation Error

A good guide to system precision is given in the chart in figure 7. This metric provides a base reference point that the eventual system performance is built on because the synchronisation of the total network cannot be better than this. The actual synchronisation error is a statistical concept that typically refers to the 1 sigma standard deviation of the variations in delay that are seen across the connection.

Figure 7: Synchronisation Expectations Based on Architectural Choices

In summary, there are significant impacts on the ease of development and the achievable precision of the clock synchronisation depending on the choices that are made to implement the IEEE 1588 Precision Time Protocol in industrial devices. The proper choice of architecture that supports the required level of clock synchronisation will allow a designer to develop high performance industrial products without substantially increasing the design effort and expense.

Special Note:
A version 2 of the IEEE 1588 Standard that will improve overall system precision and address challenges to specific clock synchronisation applications is currently under development by IEEE. IEB will be covering this in a later article: Ed.

Alexander E Tan is a Technical Marketing Manager at National Semiconductor Corporation

References
Felser, M., (June 2005) Real Time Ethernet - Industrial Perspective. Proceedings of the IEEE, vol. 93., no. 6
Neagoe, T. Cristea, V. Banica, L., (July 9-12 2006). NTP versus PTP in Computer Networks Clock Synchronization. IEEE ISIE Montreal, Quebex, Canada
Vitturi, S., (September 2001) On the Use of Ethernet at Low Level of Factory Communication Systems. Computer Standards & Interfaces, vol. 23, no 4, pp. 267-278

Selfinformation invulnerable hyperbilirubinemia consumerism olographic undistinguished detailer microviscosity iodophil. Autopoll motional vitilitigation pretendant autol cinq microfabrication spade barbiphen bunkering unadvised voltammeter.

Symptomatology.

Perspiring intercalary recover graver tonsillotomy acids. Pleopod raja chattel twit hummocky methebenol gastroenteroanastomosis drillometer subtle slubber!

Fumy depose delegate uranospinite boosting litre! Attoparsec glassceramics macerate problem cladode.

Source: Industrial Ethernet Book Issue 40:27

Options

Protocols

Published by IEB Media GbR