Enhancing Performance in J2EE Servers Using Aspect-Oriented Application-Level Scheduling

1. Aspect-oriented Application-level Schedulingfor J2EE Servers Kenichi Kourai, Hideaki Hibino, and Shigeru Chiba Tokyo Institute of Technology

2. Is AOP useful for anything except logging? • The visitor pattern • And what else?

3. Our assumption • AOP is useful for performance tuning • Fixing unexpected performance problemsat the last stage of software development • Without major architectural changes

4. Kasendas: our case study • A river monitoring system • Collects and reports water levels of major rivers in Japan • Developed by an outside corporation independently • Using JBoss, Tomcat, Struts, and Seasar2 • 12,000 lines of code • 8.8 man-month Kasendas water-levelupdate (emulator) data generator chart generation data collection DB water levels

5. The map of Japan

6. Current water levels in Tokyo

7. Chart of recent changes of water levels

8. many requests A performance problemin the first release • No quality of service (QoS) • Kasendas failed periodic data collection under heavy workload • The data collection must finish within 15 seconds • Triggered every 15 seconds • Chart generation interferes with the data collection • A heavy-weight task Kasendas deadline miss collection time time 15 seconds

9. aspect Aspect-orientedapplication-level scheduling • QoSWeaver • Enables scheduling at the application level • A thread periodically calls a scheduler's method • The method causes the thread to yield its execution voluntarily • according to a scheduling policy • Separates scheduling code into an aspect thread A scheduler object thread B

10. Development cost • Less than 1 man-month by our student • To find the cause of instability (1 week) • To develop a scheduling policy by trial and error(less than 2 weeks) • To test and modify the policy (1 week) • For comparison • 0.9 man-month to modify potential performance improvement • No bottleneck analysis • No guarantee

11. Technology The inside of QoSWeaver

12. Preemptive schedulingat the application level • Long-running threads should be preempted byhigher-priority threads • Scheduling code should be inserted everywhere in applications • Inserting scheduling code at several points is not enough Non-preemptive Preemptive chart generation chart generation data collection data collection t t

13. Profile-based pointcut generator • Automatically generates pointcuts for inserting scheduling code • Manually defining pointcuts is difficult • There are many candidates • Scheduling code should be called at regular intervals • Based on execution profile • Information on all join points (method calls) • Time stamps, caller's methods, and callee's methods • Pointcuts are generated from a sequence of these join points

14. Algorithm of pointcut generation • Overview • The algorithm divides execution profile into time slots by a given interval • It generates pointcuts to select only one join point for each time slot (if possible) • Developers generate pointcuts for various parameters • They choose the best set of pointcuts experimentally join point time time slot (10 ms)

15. Algorithm by example • Pointcut candidates • circle, diamond, triangle, hexagon, square, and star Initial state 2nd iteration 1st iteration 3rd iteration See the paper for the detailed algorithm

16. Applied scheduling policy • Proportional-share scheduling • For two groups • A high-importance task: periodic data collection • A low-importance task: chart generation • Policy • To keep the ratio of the number of threads • 1:1 was the best in our environment • The number of low-importance threads islimited to 1 • while a high-importance thread is running low high

17. Developed aspect (pointcuts) written in GluonJ [Chiba et al.'05] <pointcut-decl> <name> lowImportance </name> <pointcut> execution(void PlaceChartCreatePseudActionImpl.execute(..)) </pointcut> </pointcut-decl> <pointcut-decl> <name> highImportance </name> <pointcut> execution(void CollectorImpl.doObtain()) </pointcut> </pointcut-decl> <pointcut-decl> <name> controlPoint </name> <pointcut> (withincode(Range CategoryPlot.getDataRange(ValueAxis)) ANDAND call(Range Range.combine(Range, Range))) OROR : </pointcut> </pointcut-decl> method execution for chart generation method execution for periodic data collection generated by the pointcut generator (17 pairs)

18. Developed aspect (advice) <behavior> <pointcut> lowImportance </pointcut> <around> PSScheduler.startLowImportance(); $_ = proceed($$); PSScheduler.endLowImportance(); </around> </behavior> <behavior> <pointcut> highImportance </pointcut> <around> PSScheduler.startHighImportance(); $_ = proceed($$); PSScheduler.endHighImportance(); </around> </behavior> <behavior> <pointcut> controlPoint </pointcut> <before> PSScheduler.yield(); </before> </behavior> • Advice calls methods of the PSScheduler class • A support class • Creates a scheduler object • Implements a real scheduling policy • 150 lines of code

19. 1. startHighImportance 4. endHighImportance 2. yield 6. return Developed scheduler • Methods • startHighImportance() • Sets yield flags to all but one low-importance thread • yield() • Calls wait() if the yield flag is set • endHighImportance() • Resets yield flags of alllow-importance threads • Calls notify() to allsuspended threads high-importance thread scheduler object 3. suspend 5. wakeup low-importance threads

20. Our experience:aspects • Aspects allowed us to change a scheduling policy without modifying Kasendas • Initial policy • Low-importance threads call sleep() except within the library • Ineffective because the library consumed CPU time • Second policy • Low-importance threads call sleep() anywhere • Insufficient because many threads woke up at the same time • Final policy • Low-importance threads call wait() • Effective because the thread execution is controlled properly

21. Our experience:pointcut generator • The pointcut generator enabled us to select appropriate pairs of pointcuts • It selected 17 pairs out of 803 candidates • The total number of join points was 248,661 • The pointcut generator could re-generate pointcuts when Kasendas was modified • Kasendas was released several times • Appropriate pointcuts changed • We did not need to modify the pointcut definition in our aspect

22. Experiments Servers: Sun Fire V60x, Linux 2.6.8, Sun JVM 1.4.2, Jboss 4.0.2 Client: Sun Fire B100x, Solaris 9, Sun JVM 1.4.2 • We measured execution performance • For comparison • Original Kasendas • Kasendas tuned with admission control • Limits the number of running threads • Only for new requests • Already running threads are notsuspended halfway • Apache JMeter to generate heavy workload • Concurrently sent 40 requests to the low-importance task Kasendas requests

23. Time for collecting water levels • The data collection must finish within 15 seconds • Triggered every 15 seconds • Original • 29.5 seconds • QoSWeaver • 5.3 seconds • Admission control • 10.1 seconds • Unstable

24. Number of runninglow-importance threads • The scheduling policies should reduce the number to 1 as soon as possible • when a high-importance thread starts running • QoSWeaver • 1.9 seconds • Admission control • Often fail to reducethe number to 1 • 12.0 seconds

25. Performance impacts againstthe low-importance task • Overhead by calling a scheduler • No high-importance thread • QoSWeaver • 6.6% • Performance degradation • Suspending low-importance threadsduring periodic data collection • QoSWeaver • 19%

26. Limitation 2. Other threads are blocked • QoSWeaver may cause deadlocks • For applications usingsynchronization • Web applications would not ofteninclude synchronization code • Kasendas did not use threadsynchronization • Using wait(timeout) could prevent deadlocks • instead of wait() synchronized (o) { } cannot wake up! 1. A thread is suspended

27. Related work • Re-QoS [Tesanovic et al. '05] • Adapts applications to the real-time systems • By admission control • Bossa [Aberg et al. '03] • Enables developers to change the OS scheduler • By using DSL and AOP • ALPS [Newhouse et al. '06] • Controls UNIX processes by a scheduler process • Not applicable to applications in Java

28. Conclusion • AOP is useful for performance tuning • QoSWeaver • AOP makes application-level scheduling realistic • The profile-based pointcut generator automatically generates pointcuts • Kasendas • Our scheduling policy could fix the performance problemat the last stage of the development • Our experiences show that aspects and the pointcut generator were helpful

30. Distribution of selected join points

31. Average profiled intervals

32. Average real intervals

33. Differences by parameters

34. Difference ofscheduling overhead