Performance Evaluation of Real-Time Operating Systems

1. Performance Evaluationof Real-Time Operating Systems RTEMS, RTLinux, eCos

2. Real-Time Systems • Systems that interact predictably with events in the outside world • Examples: • Flight control systems • Collision avoidance systems • Satellite guidance systems • Patient Monitoring System • Control Systems for synchrotrons

3. Real-Time Operating Systems • Free the applications programmer from writing code for task scheduling and dispatching. • Manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints. • Widely used in all kinds of applications both on PCs and in embedded systems. • Generally operate unattended -- consequences of failure can be catastrophic.

4. What is available? • Many RTOSs have been developed: • Commercial • Open source • RTOS vendors and developers publish performance metrics to showcase their products. However, their evaluations are: • not performed on a common platform • not comprehensive

5. Evaluation of an RTOS • How would you choose an appropriate RTOS for your application? • Need for an impartial evaluation. • Several such projects have been conducted. • Only evaluated some performance characteristics • Only looked at a few commercial operating systems • Many are now out of date • ‘Real-Time Consult’ introduced a more comprehensive project. • Developed a thorough test methodology • Evaluated several commercial RTOSs • Have not evaluated open source RTOSs

6. Research Objective • Performance evaluation, on a common platform, of three open source real-time operating systems: • RTEMS • RTLinux • eCos

7. Task Scheduling Task Synchronization Task Dispatching Real-Time Operating Systems Overview RTOS

8. Last Arrival Order Task 3 Task 3 Task 2 Task 2 Task 1 First Time Task Scheduling Time Slicing Task 1

9. High Task 3 Priority Task 2 Task 2 Low Time Task Scheduling Preemptive Priority Task 1 Task 1

10. A Typical Task Control Block Program Counter Task Status Task ID Number Register Contents Pointer to next TCB : Priority Other Context Task Dispatching • Task-control blocks is the most popular method of identifying and managing tasks. • Task-Control Block (TCB) contains: • a context (e.g. program counter and register contents) • an identification string • a status, such as ready, executing or blocked • a priority (if applicable)

11. Task Synchronization • Events • Messages • Semaphores • Mutexes

12. Desired Performance Characteristics • Tasks should run with: • minimal event latency (the time between the triggering of a task and its start) • minimal jitter (the variation in running times of a task that is supposed to run at a fixed period).

13. Performance Evaluation • Throughput – speed at which the system executes instructions • Responsiveness – how fast it starts to handle in interrupt request • Determinism – how it reacts under load • how long does it take to finish what it is doing and start handling the interrupt request

14. Performance Metrics Low Level Tests • Context switching • Interrupt latency • Exclusion objects • Semaphores • Mutexes • Synchronization events

15. Switch to Task B Context Switch Time Time PC PC Register 0 Register 0 : : : : Other Context Other Context Context Switch Time Save context of Task A Load context of Task B Task A Start Task B

16. Other Considerations • Not an isolated event • Affected by number of tasks pending • Depends on priorities of pending tasks • Amount of context to be saved

17. First Instruction in Interrupt Service Routine (ISR) is executed Interrupt Occurs Task resumes ISR ends ISR Task Time Interrupt Latency Interrupt Dispatch Latency Interrupt Latency Interrupt and Interrupt Dispatch Latencies Task

18. Multiple Interrupt Latencies Interrupt-to-Task Run Interrupt Interrupt Dispatch Time Interrupt Service Routine Other Interrupt Pre-emption Disabled Scheduling Context Switch Return from System Call Task Run

19. Interrupt Task Response Time • Real response to an interrupt often occurs in a task synchronized by, but outside of, the actual ISR. • Task response also depends on whether the kernel is preemptable. If not, then kernel call must be completed before ISR.

20. Performance Metrics High Level Tests • Network throughput • Stress tests

21. Real-Time Consult Project • Evaluation and comparison of RTOS performance • Commercial systems only, to date • Evaluations available for: • Intime 1.20 • RTX 4.2 • Hyperkernel 4.3 • VxWorks/x86 5.3.1 • pSOSystem/x86 2.2.6 • QNX 6.1 • CE 3.0

22. Real-Time Consult Project • Test Method: timing is measured using an external PCI bus analyzer. • Two types of tests: • Performance Tests • Stress Tests • Qualitative Evaluation (API richness, etc.)

23. Real-Time Consult Project • Performance Tests (context switch & latencies): • Thread switch latency • time required to preempt current thread • Interrupt latency • preempt current thread and start interrupt handler • Interrupt dispatch latency • switch from interrupt context to context of interrupted thread or next thread in queue • Thread creation and deletion

24. Task start TSK0 Task end TSK0 Task start TSKN Each Task Create tasks TSK1, …, TSKN Yes Delete tasks TSK1, …, TSKN End of Test? Write trace on PCI Bus TRC_TSKN_TSN No Write trace on PCI Bus TRC_TSK0_TS0 Yield Processor Yield Processor Task Switch Latency Test

25. Real-Time Consult Project • Performance Tests (objects, file, network): • Synchronization objects • time to create and delete a synchronization object • Exclusion objects • time to create and delete an exclusion object • File system operations • creating and deleting files, reading from files and writing to files in synchronous mode • Network stack • performance of TCP/IP stack • bandwidth for various packet sizes, and CPU usage

26. Real-Time Consult Project • Stress Tests: • Two simultaneous interrupts • Interrupt nesting • maximum sustainable interrupt frequency • maximum number of objects • memory leaks

27. Focus of my Research • Evaluation of three Open Source Real-Time Operating Systems: • RTEMS Real Time Executive for Multiprocessor Systems • RTLinux Real Time Linux • eCos Embedded Configurable Operating System

28. RTEMS • Developed by the U.S. military as alternative to using commercial RTOS • Small, easy to port • High level of user configurability • Kernel is preemptible Application Dependent Software Standard Application Components Device Drivers R T E M S Target Hardware

29. RTLinux • Abstraction layer between the hardware and the standard Linux kernel • Appears as actual hardware to standard Linux kernel • Lowest priority is assigned to standard Linux kernel, which then runs as a independent task. • RTLinux executive is nonpreemptible • Leaves Linux kernel essentially untouched so it doesn’t hinder future Linux development

30. ECOS • Targeted at high-volume applications: • consumer electronics, telecommunications, automotive • other deeply embedded applications. • Configurable: • lets developer configure a system that best matches the needs of the application. • Typical configuration options: • type of scheduler • number of priority levels. • current release of the system has over 200 options • Portable • Fully preemptable

31. Typical Application • Control system for the Canadian Light Source (CLS) uses an open source real-time operating system and control software: • RTOS  RTEMS • Control Software  EPICS (Experimental Physics and Industrial Control System)