Implementing a NTP-Based Time Service within a Distributed Middleware System

1. Implementing a NTP-Based Time Service within a Distributed Middleware System ACM International Conference on the Principles and Practice of Programming in Java (PPPJ `04) Hasan Bulut Community Grids Lab. Indiana University - hbulut@indiana.edu

2. Motivation • Collaboration environments make use of distributed middleware systems to achieve collaboration among people geographically distributed. • Time ordering of events generated by entities within a distributed infrastructure is far more difficult than time ordering of events generated by a group of entities having access to the same underlying clock. • The Network Time Protocol has been around for more than a decade and being used in the Internet to let people to synchronize their computer clocks with atomic time servers clocks. • So we would like to achieve time-ordering of events generated at different entities by providing a time service to the distributed middleware system. Community Grids Lab. Indiana University - hbulut@indiana.edu

3. Example Application • An archiving system designed to archive the streams generated in a audio/video conferencing session(s) • In a collaboration session, streams from these sources need to be synchronized for archival and replay purposes. • Real-time constraints for audio/video conferencing applications can vary anytime between 30-100 msec, depending on the jitter and in inter-packet arrivals. • In a distributed event brokering environment, RTP packets are considered as events. • Time-ordering of these events are necessary to achieve an efficient archival/replay mechanism. Community Grids Lab. Indiana University - hbulut@indiana.edu

4. Computer Clocks I • Computer timers keep time by using a quartz crystal and a counter. • Two types of clocks on a computer: hardware and software. • What causes computer clocks get out of sync? • run at different rates • manufacturing defects • environment: temperature change, electric and magnetic interference • CPU load • ill-behaved program • So how much do they get out of sync? Community Grids Lab. Indiana University - hbulut@indiana.edu

5. Computer Clocks II • darya.ucs.indiana.edu - Wed Mar 17 18:29:13 EST 2004 • kamet.ucs.indiana.edu - Wed Mar 17 18:18:03 EST 2004 Community Grids Lab. Indiana University - hbulut@indiana.edu

6. Approaches to Order Events and Synchronize Clocks • Approaches can be divided into two categories: software (SW) and hardware (HW) • Logical timestamps (SW) • Lamport timestamps • Vector clocks • Synchronizing system clocks (SW) • Cristian’s algorithm • Berkeley algorithm • Averaging algorithms (decentralized algorithms) • Network Time Interface (NTI) M-Module – custom VLSI chip with interfaces to GPS receivers. (HW) Community Grids Lab. Indiana University - hbulut@indiana.edu

7. Network Time Protocol (NTP) • NTP is used to synchronize timekeeping among a set of distributed time servers and clients. • Formal specification for version 3 can be found in RFC-1305 http://www.ietf.org/rfc/rfc1305.txt • It defines the architectures, algorithms, entities and protocols used by NTP. • NTP is built on UDP/IP, which provides a connectionless mechanism. Community Grids Lab. Indiana University - hbulut@indiana.edu

8. NTP (con’t) Atomic time clock (Stratum-0) Stratum-1 time servers Stratum-2 time servers or clients • Stratum number is an integer indicating the distance from the reference clock. • Primary reference sources are synchronized by wire or radio. Community Grids Lab. Indiana University - hbulut@indiana.edu

9. T3 T2 B A tAB tBA T1 T4 NTP (con’t) • A: Client B: Time Server • Transmission time from A to B (tAB) is tAB = T2 – T1 • Transmission time from A to B (tBA) is tBA = T4 – T3 Roundtrip delay (δ) and offset (Ө ) can be computed as follows • Roundtrip delay (δ) δ = (T4 - T1) - (T3 - T2) • Offset (Ө) Ө = [(T2 - T1) + (T3 - T4)] /2 Community Grids Lab. Indiana University - hbulut@indiana.edu

10. NaradaBrokering (NB) • An event brokering middleware. Java Message Service (JMS) compliant • Supports publish-subscribe messaging model with a dynamic collection of brokers. • Provides services for TCP, UDP, Multicast, SSL, HTTP, HTTPS, raw RTP clients and storage services - SQL and file based. • Supports audio-video conferencing, communication through firewalls and proxies, software multicast. • Publish/subscribe of A/V • A A/V stream is regarded as a “topic” and each RTP packet from this stream is regarded as an “event” for this topic. • Only the sender of this stream can “publish” A/V events to this topic. Other endpoints need to subscribe to this topic in order to receive the stream. • http://www.naradabrokering.org Community Grids Lab. Indiana University - hbulut@indiana.edu

11. NB Time Service • NB Time Service runs in the kernel of NaradaBrokering. • There are two important parameters • Number of time servers • Time interval that the NB Time Service should run • A number of time servers can be specified. There is no restriction. • By default, NB Time Service uses 8 stratum-1 time servers and time interval for the service is set to 30 seconds, which means NB Time Service updates the offset every 30 seconds. Community Grids Lab. Indiana University - hbulut@indiana.edu

12. NB Time Service (con’t) • Unlike NTP daemons, NB Time Service does not change underlying system clock.Because • changing underlying system clock requires administrator privileges • the objective of this time service is to provide a mechanism to be able to time-order the events generated within NaradaBrokering. • Provides an interface, getTimestamp(), to get updated time in milliseconds. • Entities generating events in the system should utilize Time Service to timestamp the events. Community Grids Lab. Indiana University - hbulut@indiana.edu

13. Network 1 WA offset1 offset2 1 2 m 1 2 m 1 2 m BaseTime k BaseTime LocalTime adjusted time Phase 1: collecting samples Phase 2: filtering & adding to the queue Phase 3: establishing candidate list & computing offset offset1: Offset computed with regard to local computer time offset2: Offset computed with regard to adjusted time sample 1 sample 2 sample n NB Time Service (con’t) Community Grids Lab. Indiana University - hbulut@indiana.edu

14. NB Time Service (con’t) • Phase 1: Getting samples from NTP time servers. • NTP messages are sent to the time servers provided one by one. • UDP connection is used to send these messages and the timeout is set to 500 ms. • Phase 2: Filtering and adding to the queue. • Timestamps are checked to validate the received NTP reply message. • For each time server a separate queue is provided to keep the previous NTP samples. Queue length is finite. First-In-First-Out (FIFO) scheme is used to accommodate new samples when this queue is full. Community Grids Lab. Indiana University - hbulut@indiana.edu

15. NB Time Service (con’t) • Phase 3: Selection and combine algorithms. • Using clustering algorithms a candidate list is constructed. • Clustering algorithm uses synchronization distance related to each time server to find the candidate list. It is a parameter computed from roundtrip delay and dispersion. • Updating Offset • Calculating offset as described is not sufficient to achieve time-ordering of events. • Offset is updated unless adjusted time goes backwards. Community Grids Lab. Indiana University - hbulut@indiana.edu

16. Test Results I • ntpd daemon is running and it synchronizes its time with “time.nist.gov” time server. • Out of 5690 samples only 24 of them are different than zero for this experiment. Community Grids Lab. Indiana University - hbulut@indiana.edu

17. Test Results II • The first offset value is -139895 ms, which shows how much the clock in that machine is ahead of the real time. • The change of offsets is between (-3) - (2) ms. Community Grids Lab. Indiana University - hbulut@indiana.edu

18. T3 T2 B A tAB tBA T1 T4 Inter Client Discrepancy • Client discrepancy measures how much clocks of these clients are consistent with each other. • tAB and tBA can be ignored if roundtrip delay is very small. • If tAB and tBA are ignored (or if they are approximately equal to each other), the discrepancy (∆T) between these two clocks can be approximated as ∆T=0.5*(T2+T3-T1-T4) Community Grids Lab. Indiana University - hbulut@indiana.edu

19. Inter Client Discrepancy (con’t) Test duration is 143 minutes Absolute maximum discrepancy is 2 msec Absolute minimum discrepancy is 0 msec The roundtrip delay between two machines is around 2 – 4 msec. Average discrepancy for this test is 0.082 msec. Community Grids Lab. Indiana University - hbulut@indiana.edu

20. Conclusion • Different machines with different platforms and with different loads may have different clock rates which in time would get out of sync. • One cannot rely on the underlying clock and use the system clock to timestamp the events generated in messaging systems. • Real-time constraints for collaboration environments is around 100 msec. NTP provides a sufficient synchronization range for such a collaboration environment. • Stratum-1 servers should be chosen so that the roundtrip time between time servers and the local machine is minimized. • A Buffering Service has been implemented for NaradaBrokering which enables total order and time-ordered delivery. Community Grids Lab. Indiana University - hbulut@indiana.edu